2026-02-19T13:07:05.207 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-19T13:07:05.225 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-19T13:07:05.299 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/nmordech-2026-02-19_12:10:46-rados-wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-distro-default-trial/58710 branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle description: rados/standalone/{supported-random-distro$/{rpm_latest} workloads/erasure-code} email: null first_in_suite: false flavor: default job_id: '58710' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: nmordech-2026-02-19_12:10:46-rados-wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 10 os_type: rocky os_version: '10.1' overrides: admin_socket: branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug ms: 1 debug osd: 20 flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: distro-suffix: rockylinux-10 install: ceph: flavor: default sha1: 2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 selinux: allowlist: - comm="systemd".*denied.*\{ prog_run \}.*tclass=bpf.*permissive=1 workunit: branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-suite sha1: 624b9a7bcb2064ce17206482df6a1da007ff7675 owner: scheduled_nmordech@soko04.front.sepia.ceph.com priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 1255 sha1: 2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-suite suite_path: /home/teuthworker/src/github.com_NitzanMordhai_ceph_624b9a7bcb2064ce17206482df6a1da007ff7675/qa suite_relpath: qa suite_repo: https://github.com/NitzanMordhai/ceph.git suite_sha1: 624b9a7bcb2064ce17206482df6a1da007ff7675 targets: trial111.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNUVBc+qzBmmMS0gRJRo7iq8oMUsYtVbDAfuINjhdLtvEyb3YsjXktjVCqFxbt4UunC3J/evMKe9PP4aSH1KUds= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install nvmetcli nvme-cli -y - install: null - workunit: basedir: qa/standalone clients: all: - erasure-code teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-19_12:10:46 tube: trial user: nmordech verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-19T13:07:05.299 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_NitzanMordhai_ceph_624b9a7bcb2064ce17206482df6a1da007ff7675/qa; will attempt to use it 2026-02-19T13:07:05.299 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_NitzanMordhai_ceph_624b9a7bcb2064ce17206482df6a1da007ff7675/qa/tasks 2026-02-19T13:07:05.299 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-19T13:07:05.301 INFO:teuthology.task.internal:Checking packages... 2026-02-19T13:07:05.301 INFO:teuthology.task.internal:Checking packages for os_type 'rocky', flavor 'default' and ceph hash '2fc421c83f0b86ea21b9e4289cad0f7957e09eb1' 2026-02-19T13:07:05.301 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-19T13:07:05.301 INFO:teuthology.packaging:ref: None 2026-02-19T13:07:05.301 INFO:teuthology.packaging:tag: None 2026-02-19T13:07:05.301 INFO:teuthology.packaging:branch: wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle 2026-02-19T13:07:05.301 INFO:teuthology.packaging:sha1: 2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 2026-02-19T13:07:05.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&ref=wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle 2026-02-19T13:07:05.438 INFO:teuthology.task.internal:Found packages for ceph version 20.2.0-720.g2fc421c8 2026-02-19T13:07:05.440 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-19T13:07:05.453 INFO:teuthology.task.internal:no buildpackages task found 2026-02-19T13:07:05.453 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-19T13:07:05.467 INFO:teuthology.task.internal:Saving configuration 2026-02-19T13:07:05.477 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-19T13:07:05.489 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-19T13:07:05.570 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial111.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-19_12:10:46-rados-wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-distro-default-trial/58710', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'rocky', 'os_version': '10.1', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-19 13:03:31.298474', 'locked_by': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNUVBc+qzBmmMS0gRJRo7iq8oMUsYtVbDAfuINjhdLtvEyb3YsjXktjVCqFxbt4UunC3J/evMKe9PP4aSH1KUds='} 2026-02-19T13:07:05.570 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-19T13:07:05.582 INFO:teuthology.task.internal:roles: ubuntu@trial111.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-19T13:07:05.582 INFO:teuthology.run_tasks:Running task console_log... 2026-02-19T13:07:05.667 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7063344f98a0>, signals=[15]) 2026-02-19T13:07:05.667 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-19T13:07:05.679 INFO:teuthology.task.internal:Opening connections... 2026-02-19T13:07:05.679 DEBUG:teuthology.task.internal:connecting to ubuntu@trial111.front.sepia.ceph.com 2026-02-19T13:07:05.679 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T13:07:05.742 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-19T13:07:05.754 DEBUG:teuthology.orchestra.run.trial111:> uname -m 2026-02-19T13:07:05.822 INFO:teuthology.orchestra.run.trial111.stdout:x86_64 2026-02-19T13:07:05.822 DEBUG:teuthology.orchestra.run.trial111:> cat /etc/os-release 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:NAME="Rocky Linux" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:VERSION="10.1 (Red Quartz)" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:ID="rocky" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:ID_LIKE="rhel centos fedora" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:VERSION_ID="10.1" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:PLATFORM_ID="platform:el10" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:PRETTY_NAME="Rocky Linux 10.1 (Red Quartz)" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:ANSI_COLOR="0;32" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:LOGO="fedora-logo-icon" 2026-02-19T13:07:05.880 INFO:teuthology.orchestra.run.trial111.stdout:CPE_NAME="cpe:/o:rocky:rocky:10::baseos" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:HOME_URL="https://rockylinux.org/" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:VENDOR_NAME="RESF" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:VENDOR_URL="https://resf.org/" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:BUG_REPORT_URL="https://bugs.rockylinux.org/" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:SUPPORT_END="2035-05-31" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:ROCKY_SUPPORT_PRODUCT="Rocky-Linux-10" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:ROCKY_SUPPORT_PRODUCT_VERSION="10.1" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:REDHAT_SUPPORT_PRODUCT="Rocky Linux" 2026-02-19T13:07:05.881 INFO:teuthology.orchestra.run.trial111.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="10.1" 2026-02-19T13:07:05.882 INFO:teuthology.lock.ops:Updating trial111.front.sepia.ceph.com on lock server 2026-02-19T13:07:05.973 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-19T13:07:05.986 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-19T13:07:05.998 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-19T13:07:05.998 DEBUG:teuthology.orchestra.run.trial111:> test '!' -e /home/ubuntu/cephtest 2026-02-19T13:07:06.015 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-19T13:07:06.027 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-19T13:07:06.027 DEBUG:teuthology.orchestra.run.trial111:> test -z $(ls -A /var/lib/ceph) 2026-02-19T13:07:06.072 INFO:teuthology.orchestra.run.trial111.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-19T13:07:06.073 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-19T13:07:06.183 INFO:teuthology.run_tasks:Running task kernel... 2026-02-19T13:07:06.204 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-19T13:07:06.204 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-19T13:07:06.204 DEBUG:teuthology.orchestra.run.trial111:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-19T13:07:06.220 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T13:07:06.221 DEBUG:teuthology.orchestra.run.trial111:> uname -r 2026-02-19T13:07:06.277 INFO:teuthology.orchestra.run.trial111.stdout:6.12.0-124.35.1.el10_1.x86_64 2026-02-19T13:07:06.277 INFO:teuthology.task.kernel:Running kernel on trial111: 6.12.0-124.35.1.el10_1.x86_64 2026-02-19T13:07:06.277 DEBUG:teuthology.orchestra.run.trial111:> sudo yum install -y kernel 2026-02-19T13:07:06.794 INFO:teuthology.orchestra.run.trial111.stdout:Rocky Linux 10 - AppStream 23 MB/s | 2.2 MB 00:00 2026-02-19T13:07:07.124 INFO:teuthology.orchestra.run.trial111.stdout:Rocky Linux 10 - BaseOS 60 MB/s | 12 MB 00:00 2026-02-19T13:07:07.646 INFO:teuthology.orchestra.run.trial111.stdout:Rocky Linux 10 - CRB 6.3 MB/s | 508 kB 00:00 2026-02-19T13:07:07.842 INFO:teuthology.orchestra.run.trial111.stdout:Extra Packages for Enterprise Linux 43 MB/s | 6.2 MB 00:00 2026-02-19T13:07:08.504 INFO:teuthology.orchestra.run.trial111.stdout:lab-extras 300 kB/s | 3.7 kB 00:00 2026-02-19T13:07:08.916 INFO:teuthology.orchestra.run.trial111.stdout:Package kernel-6.12.0-124.27.1.el10_1.x86_64 is already installed. 2026-02-19T13:07:08.916 INFO:teuthology.orchestra.run.trial111.stdout:Package kernel-6.12.0-124.35.1.el10_1.x86_64 is already installed. 2026-02-19T13:07:08.927 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:07:08.928 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:07:08.928 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:07:08.952 DEBUG:teuthology.orchestra.run.trial111:> echo no | sudo yum reinstall kernel || true 2026-02-19T13:07:09.228 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:01 ago on Thu 19 Feb 2026 01:07:08 PM UTC. 2026-02-19T13:07:09.264 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:07:09.265 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:07:09.265 INFO:teuthology.orchestra.run.trial111.stdout: Package Architecture Version Repository Size 2026-02-19T13:07:09.265 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:07:09.265 INFO:teuthology.orchestra.run.trial111.stdout:Reinstalling: 2026-02-19T13:07:09.265 INFO:teuthology.orchestra.run.trial111.stdout: kernel x86_64 6.12.0-124.27.1.el10_1 baseos 1.4 M 2026-02-19T13:07:09.265 INFO:teuthology.orchestra.run.trial111.stdout: kernel x86_64 6.12.0-124.35.1.el10_1 baseos 1.4 M 2026-02-19T13:07:09.265 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:07:09.265 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:07:09.266 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:07:09.266 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:07:09.266 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 2.8 M 2026-02-19T13:07:09.266 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 0 2026-02-19T13:07:09.266 INFO:teuthology.orchestra.run.trial111.stderr:Operation aborted. 2026-02-19T13:07:09.284 INFO:teuthology.orchestra.run.trial111.stdout:Is this ok [y/N]: 2026-02-19T13:07:09.285 DEBUG:teuthology.orchestra.run.trial111:> sudo yum reinstall -y kernel || true 2026-02-19T13:07:09.566 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:01 ago on Thu 19 Feb 2026 01:07:08 PM UTC. 2026-02-19T13:07:09.600 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout: Package Architecture Version Repository Size 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout:Reinstalling: 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout: kernel x86_64 6.12.0-124.27.1.el10_1 baseos 1.4 M 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout: kernel x86_64 6.12.0-124.35.1.el10_1 baseos 1.4 M 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 2.8 M 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 0 2026-02-19T13:07:09.601 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:07:09.651 INFO:teuthology.orchestra.run.trial111.stdout:(1/2): kernel-6.12.0-124.35.1.el10_1.x86_64.rpm 28 MB/s | 1.4 MB 00:00 2026-02-19T13:07:09.654 INFO:teuthology.orchestra.run.trial111.stdout:(2/2): kernel-6.12.0-124.27.1.el10_1.x86_64.rpm 26 MB/s | 1.4 MB 00:00 2026-02-19T13:07:09.654 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:07:09.654 INFO:teuthology.orchestra.run.trial111.stdout:Total 52 MB/s | 2.8 MB 00:00 2026-02-19T13:07:09.654 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:07:09.733 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:07:09.733 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:07:09.754 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:07:09.754 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:07:09.798 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:07:09.805 INFO:teuthology.orchestra.run.trial111.stdout: Reinstalling : kernel-6.12.0-124.35.1.el10_1.x86_64 1/4 2026-02-19T13:07:09.808 INFO:teuthology.orchestra.run.trial111.stdout: Reinstalling : kernel-6.12.0-124.27.1.el10_1.x86_64 2/4 2026-02-19T13:07:09.810 INFO:teuthology.orchestra.run.trial111.stdout: Cleanup : kernel-6.12.0-124.35.1.el10_1.x86_64 3/4 2026-02-19T13:07:09.922 INFO:teuthology.orchestra.run.trial111.stdout: Cleanup : kernel-6.12.0-124.27.1.el10_1.x86_64 4/4 2026-02-19T13:07:09.922 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:07:09.922 INFO:teuthology.orchestra.run.trial111.stdout:Reinstalled: 2026-02-19T13:07:09.922 INFO:teuthology.orchestra.run.trial111.stdout: kernel-6.12.0-124.27.1.el10_1.x86_64 kernel-6.12.0-124.35.1.el10_1.x86_64 2026-02-19T13:07:09.922 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:07:09.923 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:07:09.948 DEBUG:teuthology.orchestra.run.trial111:> rpm -q kernel | sort -rV | head -n 1 2026-02-19T13:07:09.985 INFO:teuthology.orchestra.run.trial111.stdout:kernel-6.12.0-124.35.1.el10_1.x86_64 2026-02-19T13:07:09.985 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 6.12.0-124.35.1.el10_1.x86_64 2026-02-19T13:07:09.985 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-19T13:07:09.986 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-19T13:07:09.986 DEBUG:teuthology.orchestra.run.trial111:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-19T13:07:10.052 INFO:teuthology.orchestra.run.trial111.stdout:ttyS1 2026-02-19T13:07:10.077 DEBUG:teuthology.parallel:result is None 2026-02-19T13:07:10.077 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-19T13:07:10.091 INFO:teuthology.task.internal:Creating test directory... 2026-02-19T13:07:10.091 DEBUG:teuthology.orchestra.run.trial111:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-19T13:07:10.134 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-19T13:07:10.146 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-19T13:07:10.164 INFO:teuthology.task.internal:Creating archive directory... 2026-02-19T13:07:10.164 DEBUG:teuthology.orchestra.run.trial111:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-19T13:07:10.197 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-19T13:07:10.209 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-19T13:07:10.209 DEBUG:teuthology.orchestra.run.trial111:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-19T13:07:10.259 INFO:teuthology.orchestra.run.trial111.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:07:10.269 INFO:teuthology.orchestra.run.trial111.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:07:10.270 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-19T13:07:10.283 INFO:teuthology.task.internal:Configuring sudo... 2026-02-19T13:07:10.283 DEBUG:teuthology.orchestra.run.trial111:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-19T13:07:10.338 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-19T13:07:10.356 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-19T13:07:10.357 DEBUG:teuthology.orchestra.run.trial111:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-19T13:07:10.395 DEBUG:teuthology.orchestra.run.trial111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T13:07:10.558 DEBUG:teuthology.orchestra.run.trial111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T13:07:10.585 DEBUG:teuthology.orchestra.run.trial111:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T13:07:10.731 DEBUG:teuthology.orchestra.run.trial111:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T13:07:10.757 DEBUG:teuthology.orchestra.run.trial111:> set -ex 2026-02-19T13:07:10.757 DEBUG:teuthology.orchestra.run.trial111:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-19T13:07:10.824 DEBUG:teuthology.orchestra.run.trial111:> sudo service rsyslog restart 2026-02-19T13:07:10.894 INFO:teuthology.orchestra.run.trial111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-19T13:07:11.432 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-19T13:07:11.446 INFO:teuthology.task.internal:Starting timer... 2026-02-19T13:07:11.446 INFO:teuthology.run_tasks:Running task pcp... 2026-02-19T13:07:11.468 INFO:teuthology.run_tasks:Running task selinux... 2026-02-19T13:07:11.486 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['comm="systemd".*denied.*\\{ prog_run \\}.*tclass=bpf.*permissive=1']} 2026-02-19T13:07:11.486 DEBUG:teuthology.orchestra.run.trial111:> sudo service auditd rotate 2026-02-19T13:07:11.517 INFO:teuthology.orchestra.run.trial111.stdout:Rotating logs: 2026-02-19T13:07:11.519 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-19T13:07:11.519 DEBUG:teuthology.orchestra.run.trial111:> /usr/sbin/getenforce 2026-02-19T13:07:11.576 INFO:teuthology.orchestra.run.trial111.stdout:Permissive 2026-02-19T13:07:11.576 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial111.front.sepia.ceph.com': 'permissive'} 2026-02-19T13:07:11.577 DEBUG:teuthology.orchestra.run.trial111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e 'comm="systemd".*denied.*\{ prog_run \}.*tclass=bpf.*permissive=1' 2026-02-19T13:07:11.633 INFO:teuthology.orchestra.run.trial111.stderr:grep: Invalid content of \{\} 2026-02-19T13:07:11.643 DEBUG:teuthology.orchestra.run:got remote process result: 2 2026-02-19T13:07:11.644 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-19T13:07:11.644 DEBUG:teuthology.orchestra.run.trial111:> sudo /usr/sbin/setenforce permissive 2026-02-19T13:07:11.711 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-19T13:07:11.727 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-19T13:07:11.729 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-19T13:07:11.850 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-19T13:07:11.857 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-19T13:07:11.857 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial111.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-19T13:08:11.813 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial111.front.sepia.ceph.com')] 2026-02-19T13:08:11.815 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial111.front.sepia.ceph.com' 2026-02-19T13:08:11.817 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial111.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-19T13:08:11.904 DEBUG:teuthology.orchestra.run.trial111:> true 2026-02-19T13:08:11.993 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial111.front.sepia.ceph.com' 2026-02-19T13:08:11.993 INFO:teuthology.run_tasks:Running task clock... 2026-02-19T13:08:12.010 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-19T13:08:12.011 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-19T13:08:12.011 DEBUG:teuthology.orchestra.run.trial111:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T13:08:12.063 INFO:teuthology.orchestra.run.trial111.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-19T13:08:12.075 INFO:teuthology.orchestra.run.trial111.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-19T13:08:12.098 INFO:teuthology.orchestra.run.trial111.stderr:sudo: ntpd: command not found 2026-02-19T13:08:12.107 INFO:teuthology.orchestra.run.trial111.stdout:506 Cannot talk to daemon 2026-02-19T13:08:12.120 INFO:teuthology.orchestra.run.trial111.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-19T13:08:12.132 INFO:teuthology.orchestra.run.trial111.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-19T13:08:12.176 INFO:teuthology.orchestra.run.trial111.stderr:bash: line 1: ntpq: command not found 2026-02-19T13:08:12.181 INFO:teuthology.orchestra.run.trial111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-19T13:08:12.182 INFO:teuthology.orchestra.run.trial111.stdout:=============================================================================== 2026-02-19T13:08:12.182 INFO:teuthology.orchestra.run.trial111.stdout:^? 172-235-154-118.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-19T13:08:12.182 INFO:teuthology.run_tasks:Running task pexec... 2026-02-19T13:08:12.197 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-19T13:08:12.197 DEBUG:teuthology.orchestra.run.trial111:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-19T13:08:12.225 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial111.front.sepia.ceph.com 2026-02-19T13:08:12.226 INFO:teuthology.task.pexec:sudo dnf remove nvme-cli -y 2026-02-19T13:08:12.226 INFO:teuthology.task.pexec:sudo dnf install nvmetcli nvme-cli -y 2026-02-19T13:08:12.407 INFO:teuthology.orchestra.run.trial111.stdout:No match for argument: nvme-cli 2026-02-19T13:08:12.407 INFO:teuthology.orchestra.run.trial111.stderr:No packages marked for removal. 2026-02-19T13:08:12.409 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:08:12.409 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:08:12.410 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:08:12.690 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:27 ago on Thu 19 Feb 2026 01:07:45 PM UTC. 2026-02-19T13:08:12.725 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:08:12.725 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:12.725 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repository Size 2026-02-19T13:08:12.725 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:12.725 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: nvme-cli x86_64 2.13-2.el10 baseos 1.1 M 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: nvmetcli noarch 0.8-1.el10 baseos 54 k 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout:Installing dependencies: 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: python3-configshell noarch 1:1.1.30-8.el10 baseos 90 k 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: python3-kmod x86_64 0.9.2-6.el10 baseos 111 k 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: python3-pyparsing noarch 3.1.1-7.el10 baseos 271 k 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: python3-typing-extensions noarch 4.9.0-6.el10 baseos 77 k 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: python3-urwid x86_64 2.5.3-4.el10 baseos 1.1 M 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: python3-wcwidth noarch 0.2.6-6.el10 baseos 48 k 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout:Install 8 Packages 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 2.8 M 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 13 M 2026-02-19T13:08:12.726 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:08:12.766 INFO:teuthology.orchestra.run.trial111.stdout:(1/8): nvmetcli-0.8-1.el10.noarch.rpm 1.3 MB/s | 54 kB 00:00 2026-02-19T13:08:12.767 INFO:teuthology.orchestra.run.trial111.stdout:(2/8): python3-configshell-1.1.30-8.el10.noarch 2.2 MB/s | 90 kB 00:00 2026-02-19T13:08:12.774 INFO:teuthology.orchestra.run.trial111.stdout:(3/8): nvme-cli-2.13-2.el10.x86_64.rpm 23 MB/s | 1.1 MB 00:00 2026-02-19T13:08:12.784 INFO:teuthology.orchestra.run.trial111.stdout:(4/8): python3-kmod-0.9.2-6.el10.x86_64.rpm 6.2 MB/s | 111 kB 00:00 2026-02-19T13:08:12.785 INFO:teuthology.orchestra.run.trial111.stdout:(5/8): python3-pyparsing-3.1.1-7.el10.noarch.rp 15 MB/s | 271 kB 00:00 2026-02-19T13:08:12.789 INFO:teuthology.orchestra.run.trial111.stdout:(6/8): python3-typing-extensions-4.9.0-6.el10.n 5.0 MB/s | 77 kB 00:00 2026-02-19T13:08:12.807 INFO:teuthology.orchestra.run.trial111.stdout:(7/8): python3-wcwidth-0.2.6-6.el10.noarch.rpm 2.2 MB/s | 48 kB 00:00 2026-02-19T13:08:12.808 INFO:teuthology.orchestra.run.trial111.stdout:(8/8): python3-urwid-2.5.3-4.el10.x86_64.rpm 43 MB/s | 1.1 MB 00:00 2026-02-19T13:08:12.809 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:08:12.809 INFO:teuthology.orchestra.run.trial111.stdout:Total 34 MB/s | 2.8 MB 00:00 2026-02-19T13:08:12.809 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:08:12.817 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:08:12.818 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:08:12.884 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:08:12.884 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:08:12.954 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:08:12.957 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-wcwidth-0.2.6-6.el10.noarch 1/8 2026-02-19T13:08:12.984 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-typing-extensions-4.9.0-6.el10.noarch 2/8 2026-02-19T13:08:12.991 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-urwid-2.5.3-4.el10.x86_64 3/8 2026-02-19T13:08:12.995 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-pyparsing-3.1.1-7.el10.noarch 4/8 2026-02-19T13:08:12.999 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-configshell-1:1.1.30-8.el10.noarch 5/8 2026-02-19T13:08:13.004 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-kmod-0.9.2-6.el10.x86_64 6/8 2026-02-19T13:08:13.005 INFO:teuthology.orchestra.run.trial111.stdout: Installing : nvmetcli-0.8-1.el10.noarch 7/8 2026-02-19T13:08:13.062 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: nvmetcli-0.8-1.el10.noarch 7/8 2026-02-19T13:08:13.065 INFO:teuthology.orchestra.run.trial111.stdout: Installing : nvme-cli-2.13-2.el10.x86_64 8/8 2026-02-19T13:08:13.314 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: nvme-cli-2.13-2.el10.x86_64 8/8 2026-02-19T13:08:13.315 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/default.target.wants/nvmefc-boot-connections.service' → '/usr/lib/systemd/system/nvmefc-boot-connections.service'. 2026-02-19T13:08:13.315 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:13.637 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:13.637 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:08:13.637 INFO:teuthology.orchestra.run.trial111.stdout: nvme-cli-2.13-2.el10.x86_64 2026-02-19T13:08:13.637 INFO:teuthology.orchestra.run.trial111.stdout: nvmetcli-0.8-1.el10.noarch 2026-02-19T13:08:13.637 INFO:teuthology.orchestra.run.trial111.stdout: python3-configshell-1:1.1.30-8.el10.noarch 2026-02-19T13:08:13.637 INFO:teuthology.orchestra.run.trial111.stdout: python3-kmod-0.9.2-6.el10.x86_64 2026-02-19T13:08:13.638 INFO:teuthology.orchestra.run.trial111.stdout: python3-pyparsing-3.1.1-7.el10.noarch 2026-02-19T13:08:13.638 INFO:teuthology.orchestra.run.trial111.stdout: python3-typing-extensions-4.9.0-6.el10.noarch 2026-02-19T13:08:13.638 INFO:teuthology.orchestra.run.trial111.stdout: python3-urwid-2.5.3-4.el10.x86_64 2026-02-19T13:08:13.638 INFO:teuthology.orchestra.run.trial111.stdout: python3-wcwidth-0.2.6-6.el10.noarch 2026-02-19T13:08:13.638 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:13.638 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:08:13.665 DEBUG:teuthology.parallel:result is None 2026-02-19T13:08:13.665 INFO:teuthology.run_tasks:Running task install... 2026-02-19T13:08:13.679 DEBUG:teuthology.task.install:project ceph 2026-02-19T13:08:13.679 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2fc421c83f0b86ea21b9e4289cad0f7957e09eb1'}} 2026-02-19T13:08:13.679 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '2fc421c83f0b86ea21b9e4289cad0f7957e09eb1'} 2026-02-19T13:08:13.679 INFO:teuthology.task.install:Using flavor: default 2026-02-19T13:08:13.688 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-19T13:08:13.688 INFO:teuthology.task.install:extra packages: [] 2026-02-19T13:08:13.689 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2fc421c83f0b86ea21b9e4289cad0f7957e09eb1', 'tag': None, 'wait_for_package': False} 2026-02-19T13:08:13.690 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&sha1=2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 2026-02-19T13:08:13.857 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/rocky/10/flavors/default/ 2026-02-19T13:08:13.857 INFO:teuthology.task.install.rpm:Package version is 20.2.0-720.g2fc421c8 2026-02-19T13:08:14.181 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/rocky/10/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/rocky/10/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/rocky/10/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-19T13:08:14.182 DEBUG:teuthology.orchestra.run.trial111:> set -ex 2026-02-19T13:08:14.182 DEBUG:teuthology.orchestra.run.trial111:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-19T13:08:14.212 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-19T13:08:14.212 DEBUG:teuthology.orchestra.run.trial111:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2fc421c83f0b86ea21b9e4289cad0f7957e09eb1/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-19T13:08:14.279 DEBUG:teuthology.orchestra.run.trial111:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-19T13:08:14.357 DEBUG:teuthology.orchestra.run.trial111:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-19T13:08:14.424 INFO:teuthology.orchestra.run.trial111.stdout:check_obsoletes = 1 2026-02-19T13:08:14.426 DEBUG:teuthology.orchestra.run.trial111:> sudo yum clean all 2026-02-19T13:08:14.574 INFO:teuthology.orchestra.run.trial111.stdout:28 files removed 2026-02-19T13:08:14.589 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-radosgw 2026-02-19T13:08:15.292 INFO:teuthology.orchestra.run.trial111.stdout:ceph packages for x86_64 44 kB/s | 24 kB 00:00 2026-02-19T13:08:15.777 INFO:teuthology.orchestra.run.trial111.stdout:ceph noarch packages 8.2 kB/s | 3.9 kB 00:00 2026-02-19T13:08:16.222 INFO:teuthology.orchestra.run.trial111.stdout:ceph source packages 3.8 kB/s | 1.7 kB 00:00 2026-02-19T13:08:16.331 INFO:teuthology.orchestra.run.trial111.stdout:Rocky Linux 10 - AppStream 21 MB/s | 2.2 MB 00:00 2026-02-19T13:08:16.684 INFO:teuthology.orchestra.run.trial111.stdout:Rocky Linux 10 - BaseOS 54 MB/s | 12 MB 00:00 2026-02-19T13:08:17.204 INFO:teuthology.orchestra.run.trial111.stdout:Rocky Linux 10 - CRB 6.4 MB/s | 508 kB 00:00 2026-02-19T13:08:17.695 INFO:teuthology.orchestra.run.trial111.stdout:Extra Packages for Enterprise Linux 14 MB/s | 6.2 MB 00:00 2026-02-19T13:08:18.365 INFO:teuthology.orchestra.run.trial111.stdout:lab-extras 315 kB/s | 3.7 kB 00:00 2026-02-19T13:08:18.882 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:08:18.883 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:18.883 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repository Size 2026-02-19T13:08:18.883 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:18.883 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:08:18.884 INFO:teuthology.orchestra.run.trial111.stdout: ceph-radosgw x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 24 M 2026-02-19T13:08:18.884 INFO:teuthology.orchestra.run.trial111.stdout:Upgrading: 2026-02-19T13:08:18.884 INFO:teuthology.orchestra.run.trial111.stdout: librados2 x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 3.7 M 2026-02-19T13:08:18.884 INFO:teuthology.orchestra.run.trial111.stdout: librbd1 x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 2.9 M 2026-02-19T13:08:18.884 INFO:teuthology.orchestra.run.trial111.stdout:Installing dependencies: 2026-02-19T13:08:18.885 INFO:teuthology.orchestra.run.trial111.stdout: abseil-cpp x86_64 20240722.1-1.el10_0 epel 590 k 2026-02-19T13:08:18.885 INFO:teuthology.orchestra.run.trial111.stdout: boost-program-options x86_64 1.83.0-5.el10 appstream 109 k 2026-02-19T13:08:18.885 INFO:teuthology.orchestra.run.trial111.stdout: ceph-base x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 6.0 M 2026-02-19T13:08:18.885 INFO:teuthology.orchestra.run.trial111.stdout: ceph-common x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 24 M 2026-02-19T13:08:18.885 INFO:teuthology.orchestra.run.trial111.stdout: ceph-selinux x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 25 k 2026-02-19T13:08:18.886 INFO:teuthology.orchestra.run.trial111.stdout: daxctl-libs x86_64 80-4.el10 baseos 42 k 2026-02-19T13:08:18.886 INFO:teuthology.orchestra.run.trial111.stdout: gperftools-libs x86_64 2.15-4.el10_0 epel 303 k 2026-02-19T13:08:18.886 INFO:teuthology.orchestra.run.trial111.stdout: libarrow x86_64 17.0.0-9.el10_2 epel 5.7 M 2026-02-19T13:08:18.886 INFO:teuthology.orchestra.run.trial111.stdout: libarrow-doc noarch 17.0.0-9.el10_2 epel 27 k 2026-02-19T13:08:18.886 INFO:teuthology.orchestra.run.trial111.stdout: libcephfs2 x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 897 k 2026-02-19T13:08:18.887 INFO:teuthology.orchestra.run.trial111.stdout: libnbd x86_64 1.22.2-3.el10_1 appstream 175 k 2026-02-19T13:08:18.887 INFO:teuthology.orchestra.run.trial111.stdout: liboath x86_64 2.6.12-1.el10_0 epel 48 k 2026-02-19T13:08:18.887 INFO:teuthology.orchestra.run.trial111.stdout: liborc2 x86_64 2.0.7-1.el10_2 epel 515 k 2026-02-19T13:08:18.887 INFO:teuthology.orchestra.run.trial111.stdout: libpmem x86_64 2.1.0-2.el10_0 epel 112 k 2026-02-19T13:08:18.887 INFO:teuthology.orchestra.run.trial111.stdout: libpmemobj x86_64 2.1.0-2.el10_0 epel 161 k 2026-02-19T13:08:18.888 INFO:teuthology.orchestra.run.trial111.stdout: librabbitmq x86_64 0.14.0-3.el10 appstream 43 k 2026-02-19T13:08:18.888 INFO:teuthology.orchestra.run.trial111.stdout: libradosstriper1 x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 254 k 2026-02-19T13:08:18.888 INFO:teuthology.orchestra.run.trial111.stdout: librdkafka x86_64 2.1.1-7.el10 appstream 725 k 2026-02-19T13:08:18.888 INFO:teuthology.orchestra.run.trial111.stdout: librgw2 x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 6.6 M 2026-02-19T13:08:18.888 INFO:teuthology.orchestra.run.trial111.stdout: libunwind x86_64 1.8.0-4.el10_0 epel 73 k 2026-02-19T13:08:18.888 INFO:teuthology.orchestra.run.trial111.stdout: lttng-ust x86_64 2.13.7-5.el10 crb 334 k 2026-02-19T13:08:18.889 INFO:teuthology.orchestra.run.trial111.stdout: mailcap noarch 2.1.54-8.el10 baseos 34 k 2026-02-19T13:08:18.889 INFO:teuthology.orchestra.run.trial111.stdout: ndctl-libs x86_64 80-4.el10 baseos 89 k 2026-02-19T13:08:18.889 INFO:teuthology.orchestra.run.trial111.stdout: parquet-libs x86_64 17.0.0-9.el10_2 epel 1.0 M 2026-02-19T13:08:18.889 INFO:teuthology.orchestra.run.trial111.stdout: pciutils x86_64 3.13.0-6.el10 baseos 122 k 2026-02-19T13:08:18.889 INFO:teuthology.orchestra.run.trial111.stdout: protobuf x86_64 3.19.6-11.el10 appstream 1.0 M 2026-02-19T13:08:18.890 INFO:teuthology.orchestra.run.trial111.stdout: python3-ceph-argparse x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 57 k 2026-02-19T13:08:18.890 INFO:teuthology.orchestra.run.trial111.stdout: python3-ceph-common x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 224 k 2026-02-19T13:08:18.890 INFO:teuthology.orchestra.run.trial111.stdout: python3-cephfs x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 206 k 2026-02-19T13:08:18.890 INFO:teuthology.orchestra.run.trial111.stdout: python3-prettytable noarch 3.6.0-10.el10 appstream 70 k 2026-02-19T13:08:18.890 INFO:teuthology.orchestra.run.trial111.stdout: python3-rados x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 350 k 2026-02-19T13:08:18.891 INFO:teuthology.orchestra.run.trial111.stdout: python3-rbd x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 366 k 2026-02-19T13:08:18.891 INFO:teuthology.orchestra.run.trial111.stdout: python3-rgw x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 130 k 2026-02-19T13:08:18.891 INFO:teuthology.orchestra.run.trial111.stdout: qatlib x86_64 24.09.0-2.el10 appstream 229 k 2026-02-19T13:08:18.891 INFO:teuthology.orchestra.run.trial111.stdout: qatzip-libs x86_64 1.3.1-1.el10 appstream 66 k 2026-02-19T13:08:18.891 INFO:teuthology.orchestra.run.trial111.stdout: re2 x86_64 2:20251105-1.el10_2 epel 221 k 2026-02-19T13:08:18.891 INFO:teuthology.orchestra.run.trial111.stdout: thrift x86_64 0.20.0-4.el10_1 epel 1.7 M 2026-02-19T13:08:18.892 INFO:teuthology.orchestra.run.trial111.stdout: utf8proc x86_64 2.7.0-9.el10 appstream 80 k 2026-02-19T13:08:18.892 INFO:teuthology.orchestra.run.trial111.stdout:Installing weak dependencies: 2026-02-19T13:08:18.892 INFO:teuthology.orchestra.run.trial111.stdout: libcephfs-daemon x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 36 k 2026-02-19T13:08:18.892 INFO:teuthology.orchestra.run.trial111.stdout: libcephfs-proxy2 x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 25 k 2026-02-19T13:08:18.892 INFO:teuthology.orchestra.run.trial111.stdout: qatlib-service x86_64 24.09.0-2.el10 appstream 37 k 2026-02-19T13:08:18.892 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:18.893 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:08:18.893 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:18.893 INFO:teuthology.orchestra.run.trial111.stdout:Install 42 Packages 2026-02-19T13:08:18.893 INFO:teuthology.orchestra.run.trial111.stdout:Upgrade 2 Packages 2026-02-19T13:08:18.893 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:18.894 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 84 M 2026-02-19T13:08:18.895 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:08:20.933 INFO:teuthology.orchestra.run.trial111.stdout:(1/44): ceph-base-20.2.0-720.g2fc421c8.el10.x86 3.1 MB/s | 6.0 MB 00:01 2026-02-19T13:08:20.998 INFO:teuthology.orchestra.run.trial111.stdout:(2/44): ceph-selinux-20.2.0-720.g2fc421c8.el10. 389 kB/s | 25 kB 00:00 2026-02-19T13:08:21.063 INFO:teuthology.orchestra.run.trial111.stdout:(3/44): libcephfs-daemon-20.2.0-720.g2fc421c8.e 549 kB/s | 36 kB 00:00 2026-02-19T13:08:21.128 INFO:teuthology.orchestra.run.trial111.stdout:(4/44): libcephfs-proxy2-20.2.0-720.g2fc421c8.e 378 kB/s | 25 kB 00:00 2026-02-19T13:08:21.340 INFO:teuthology.orchestra.run.trial111.stdout:(5/44): libcephfs2-20.2.0-720.g2fc421c8.el10.x8 4.1 MB/s | 897 kB 00:00 2026-02-19T13:08:21.411 INFO:teuthology.orchestra.run.trial111.stdout:(6/44): libradosstriper1-20.2.0-720.g2fc421c8.e 3.5 MB/s | 254 kB 00:00 2026-02-19T13:08:22.767 INFO:teuthology.orchestra.run.trial111.stdout:(7/44): librgw2-20.2.0-720.g2fc421c8.el10.x86_6 4.8 MB/s | 6.6 MB 00:01 2026-02-19T13:08:22.830 INFO:teuthology.orchestra.run.trial111.stdout:(8/44): python3-ceph-argparse-20.2.0-720.g2fc42 903 kB/s | 57 kB 00:00 2026-02-19T13:08:22.897 INFO:teuthology.orchestra.run.trial111.stdout:(9/44): python3-ceph-common-20.2.0-720.g2fc421c 3.3 MB/s | 224 kB 00:00 2026-02-19T13:08:22.964 INFO:teuthology.orchestra.run.trial111.stdout:(10/44): python3-cephfs-20.2.0-720.g2fc421c8.el 3.0 MB/s | 206 kB 00:00 2026-02-19T13:08:23.034 INFO:teuthology.orchestra.run.trial111.stdout:(11/44): python3-rados-20.2.0-720.g2fc421c8.el1 4.8 MB/s | 350 kB 00:00 2026-02-19T13:08:23.166 INFO:teuthology.orchestra.run.trial111.stdout:(12/44): python3-rbd-20.2.0-720.g2fc421c8.el10. 2.7 MB/s | 366 kB 00:00 2026-02-19T13:08:23.232 INFO:teuthology.orchestra.run.trial111.stdout:(13/44): python3-rgw-20.2.0-720.g2fc421c8.el10. 1.9 MB/s | 130 kB 00:00 2026-02-19T13:08:23.270 INFO:teuthology.orchestra.run.trial111.stdout:(14/44): boost-program-options-1.83.0-5.el10.x8 2.8 MB/s | 109 kB 00:00 2026-02-19T13:08:23.289 INFO:teuthology.orchestra.run.trial111.stdout:(15/44): libnbd-1.22.2-3.el10_1.x86_64.rpm 9.0 MB/s | 175 kB 00:00 2026-02-19T13:08:23.308 INFO:teuthology.orchestra.run.trial111.stdout:(16/44): librabbitmq-0.14.0-3.el10.x86_64.rpm 2.2 MB/s | 43 kB 00:00 2026-02-19T13:08:23.336 INFO:teuthology.orchestra.run.trial111.stdout:(17/44): librdkafka-2.1.1-7.el10.x86_64.rpm 25 MB/s | 725 kB 00:00 2026-02-19T13:08:23.361 INFO:teuthology.orchestra.run.trial111.stdout:(18/44): protobuf-3.19.6-11.el10.x86_64.rpm 42 MB/s | 1.0 MB 00:00 2026-02-19T13:08:23.378 INFO:teuthology.orchestra.run.trial111.stdout:(19/44): python3-prettytable-3.6.0-10.el10.noar 4.1 MB/s | 70 kB 00:00 2026-02-19T13:08:23.395 INFO:teuthology.orchestra.run.trial111.stdout:(20/44): qatlib-24.09.0-2.el10.x86_64.rpm 13 MB/s | 229 kB 00:00 2026-02-19T13:08:23.412 INFO:teuthology.orchestra.run.trial111.stdout:(21/44): qatlib-service-24.09.0-2.el10.x86_64.r 2.1 MB/s | 37 kB 00:00 2026-02-19T13:08:23.429 INFO:teuthology.orchestra.run.trial111.stdout:(22/44): qatzip-libs-1.3.1-1.el10.x86_64.rpm 3.8 MB/s | 66 kB 00:00 2026-02-19T13:08:23.447 INFO:teuthology.orchestra.run.trial111.stdout:(23/44): utf8proc-2.7.0-9.el10.x86_64.rpm 4.5 MB/s | 80 kB 00:00 2026-02-19T13:08:23.461 INFO:teuthology.orchestra.run.trial111.stdout:(24/44): daxctl-libs-80-4.el10.x86_64.rpm 3.0 MB/s | 42 kB 00:00 2026-02-19T13:08:23.474 INFO:teuthology.orchestra.run.trial111.stdout:(25/44): mailcap-2.1.54-8.el10.noarch.rpm 2.5 MB/s | 34 kB 00:00 2026-02-19T13:08:23.493 INFO:teuthology.orchestra.run.trial111.stdout:(26/44): ndctl-libs-80-4.el10.x86_64.rpm 4.6 MB/s | 89 kB 00:00 2026-02-19T13:08:23.514 INFO:teuthology.orchestra.run.trial111.stdout:(27/44): pciutils-3.13.0-6.el10.x86_64.rpm 5.8 MB/s | 122 kB 00:00 2026-02-19T13:08:23.531 INFO:teuthology.orchestra.run.trial111.stdout:(28/44): lttng-ust-2.13.7-5.el10.x86_64.rpm 19 MB/s | 334 kB 00:00 2026-02-19T13:08:23.718 INFO:teuthology.orchestra.run.trial111.stdout:(29/44): abseil-cpp-20240722.1-1.el10_0.x86_64. 3.1 MB/s | 590 kB 00:00 2026-02-19T13:08:23.745 INFO:teuthology.orchestra.run.trial111.stdout:(30/44): gperftools-libs-2.15-4.el10_0.x86_64.r 11 MB/s | 303 kB 00:00 2026-02-19T13:08:23.849 INFO:teuthology.orchestra.run.trial111.stdout:(31/44): libarrow-17.0.0-9.el10_2.x86_64.rpm 55 MB/s | 5.7 MB 00:00 2026-02-19T13:08:23.869 INFO:teuthology.orchestra.run.trial111.stdout:(32/44): libarrow-doc-17.0.0-9.el10_2.noarch.rp 1.3 MB/s | 27 kB 00:00 2026-02-19T13:08:23.890 INFO:teuthology.orchestra.run.trial111.stdout:(33/44): liboath-2.6.12-1.el10_0.x86_64.rpm 2.3 MB/s | 48 kB 00:00 2026-02-19T13:08:23.916 INFO:teuthology.orchestra.run.trial111.stdout:(34/44): liborc2-2.0.7-1.el10_2.x86_64.rpm 20 MB/s | 515 kB 00:00 2026-02-19T13:08:23.937 INFO:teuthology.orchestra.run.trial111.stdout:(35/44): libpmem-2.1.0-2.el10_0.x86_64.rpm 5.2 MB/s | 112 kB 00:00 2026-02-19T13:08:23.959 INFO:teuthology.orchestra.run.trial111.stdout:(36/44): libpmemobj-2.1.0-2.el10_0.x86_64.rpm 7.2 MB/s | 161 kB 00:00 2026-02-19T13:08:23.980 INFO:teuthology.orchestra.run.trial111.stdout:(37/44): libunwind-1.8.0-4.el10_0.x86_64.rpm 3.4 MB/s | 73 kB 00:00 2026-02-19T13:08:24.011 INFO:teuthology.orchestra.run.trial111.stdout:(38/44): parquet-libs-17.0.0-9.el10_2.x86_64.rp 33 MB/s | 1.0 MB 00:00 2026-02-19T13:08:24.033 INFO:teuthology.orchestra.run.trial111.stdout:(39/44): re2-20251105-1.el10_2.x86_64.rpm 9.6 MB/s | 221 kB 00:00 2026-02-19T13:08:24.070 INFO:teuthology.orchestra.run.trial111.stdout:(40/44): thrift-0.20.0-4.el10_1.x86_64.rpm 45 MB/s | 1.7 MB 00:00 2026-02-19T13:08:24.851 INFO:teuthology.orchestra.run.trial111.stdout:(41/44): ceph-common-20.2.0-720.g2fc421c8.el10. 4.2 MB/s | 24 MB 00:05 2026-02-19T13:08:25.182 INFO:teuthology.orchestra.run.trial111.stdout:(42/44): librados2-20.2.0-720.g2fc421c8.el10.x8 3.3 MB/s | 3.7 MB 00:01 2026-02-19T13:08:25.541 INFO:teuthology.orchestra.run.trial111.stdout:(43/44): librbd1-20.2.0-720.g2fc421c8.el10.x86_ 4.1 MB/s | 2.9 MB 00:00 2026-02-19T13:08:27.423 INFO:teuthology.orchestra.run.trial111.stdout:(44/44): ceph-radosgw-20.2.0-720.g2fc421c8.el10 2.9 MB/s | 24 MB 00:08 2026-02-19T13:08:27.424 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:08:27.424 INFO:teuthology.orchestra.run.trial111.stdout:Total 9.8 MB/s | 84 MB 00:08 2026-02-19T13:08:27.424 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:08:27.443 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:08:27.444 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:08:27.742 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:08:27.742 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:08:28.098 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:08:28.101 INFO:teuthology.orchestra.run.trial111.stdout: Installing : lttng-ust-2.13.7-5.el10.x86_64 1/46 2026-02-19T13:08:28.107 INFO:teuthology.orchestra.run.trial111.stdout: Installing : daxctl-libs-80-4.el10.x86_64 2/46 2026-02-19T13:08:28.109 INFO:teuthology.orchestra.run.trial111.stdout: Installing : librdkafka-2.1.1-7.el10.x86_64 3/46 2026-02-19T13:08:28.111 INFO:teuthology.orchestra.run.trial111.stdout: Installing : librabbitmq-0.14.0-3.el10.x86_64 4/46 2026-02-19T13:08:28.121 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ndctl-libs-80-4.el10.x86_64 5/46 2026-02-19T13:08:28.236 INFO:teuthology.orchestra.run.trial111.stdout: Installing : thrift-0.20.0-4.el10_1.x86_64 6/46 2026-02-19T13:08:28.238 INFO:teuthology.orchestra.run.trial111.stdout: Upgrading : librados2-2:20.2.0-720.g2fc421c8.el10.x86_64 7/46 2026-02-19T13:08:28.280 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: librados2-2:20.2.0-720.g2fc421c8.el10.x86_64 7/46 2026-02-19T13:08:28.280 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libcephfs2-2:20.2.0-720.g2fc421c8.el10.x86_64 8/46 2026-02-19T13:08:28.293 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: libcephfs2-2:20.2.0-720.g2fc421c8.el10.x86_64 8/46 2026-02-19T13:08:28.295 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libcephfs-daemon-2:20.2.0-720.g2fc421c8.el10.x86_6 9/46 2026-02-19T13:08:28.296 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libcephfs-proxy2-2:20.2.0-720.g2fc421c8.el10.x86_6 10/46 2026-02-19T13:08:28.318 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: libcephfs-proxy2-2:20.2.0-720.g2fc421c8.el10.x86_6 10/46 2026-02-19T13:08:28.320 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-rados-2:20.2.0-720.g2fc421c8.el10.x86_64 11/46 2026-02-19T13:08:28.322 INFO:teuthology.orchestra.run.trial111.stdout: Installing : liboath-2.6.12-1.el10_0.x86_64 12/46 2026-02-19T13:08:28.324 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libarrow-doc-17.0.0-9.el10_2.noarch 13/46 2026-02-19T13:08:28.332 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-ceph-argparse-2:20.2.0-720.g2fc421c8.el10. 14/46 2026-02-19T13:08:28.341 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-cephfs-2:20.2.0-720.g2fc421c8.el10.x86_64 15/46 2026-02-19T13:08:28.342 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libradosstriper1-2:20.2.0-720.g2fc421c8.el10.x86_6 16/46 2026-02-19T13:08:28.352 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: libradosstriper1-2:20.2.0-720.g2fc421c8.el10.x86_6 16/46 2026-02-19T13:08:28.355 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libpmem-2.1.0-2.el10_0.x86_64 17/46 2026-02-19T13:08:28.359 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libpmemobj-2.1.0-2.el10_0.x86_64 18/46 2026-02-19T13:08:28.365 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libunwind-1.8.0-4.el10_0.x86_64 19/46 2026-02-19T13:08:28.390 INFO:teuthology.orchestra.run.trial111.stdout: Installing : gperftools-libs-2.15-4.el10_0.x86_64 20/46 2026-02-19T13:08:28.394 INFO:teuthology.orchestra.run.trial111.stdout: Installing : abseil-cpp-20240722.1-1.el10_0.x86_64 21/46 2026-02-19T13:08:28.397 INFO:teuthology.orchestra.run.trial111.stdout: Installing : re2-2:20251105-1.el10_2.x86_64 22/46 2026-02-19T13:08:28.399 INFO:teuthology.orchestra.run.trial111.stdout: Installing : pciutils-3.13.0-6.el10.x86_64 23/46 2026-02-19T13:08:28.400 INFO:teuthology.orchestra.run.trial111.stdout: Installing : qatlib-service-24.09.0-2.el10.x86_64 24/46 2026-02-19T13:08:28.415 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: qatlib-service-24.09.0-2.el10.x86_64 24/46 2026-02-19T13:08:28.431 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: qatlib-24.09.0-2.el10.x86_64 25/46 2026-02-19T13:08:28.434 INFO:teuthology.orchestra.run.trial111.stdout: Installing : qatlib-24.09.0-2.el10.x86_64 25/46 2026-02-19T13:08:28.435 INFO:teuthology.orchestra.run.trial111.stdout: Installing : qatzip-libs-1.3.1-1.el10.x86_64 26/46 2026-02-19T13:08:28.437 INFO:teuthology.orchestra.run.trial111.stdout: Installing : mailcap-2.1.54-8.el10.noarch 27/46 2026-02-19T13:08:28.440 INFO:teuthology.orchestra.run.trial111.stdout: Installing : utf8proc-2.7.0-9.el10.x86_64 28/46 2026-02-19T13:08:28.447 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-prettytable-3.6.0-10.el10.noarch 29/46 2026-02-19T13:08:28.451 INFO:teuthology.orchestra.run.trial111.stdout: Installing : protobuf-3.19.6-11.el10.x86_64 30/46 2026-02-19T13:08:28.488 INFO:teuthology.orchestra.run.trial111.stdout: Installing : liborc2-2.0.7-1.el10_2.x86_64 31/46 2026-02-19T13:08:28.491 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libarrow-17.0.0-9.el10_2.x86_64 32/46 2026-02-19T13:08:28.580 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libnbd-1.22.2-3.el10_1.x86_64 33/46 2026-02-19T13:08:28.581 INFO:teuthology.orchestra.run.trial111.stdout: Upgrading : librbd1-2:20.2.0-720.g2fc421c8.el10.x86_64 34/46 2026-02-19T13:08:28.608 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: librbd1-2:20.2.0-720.g2fc421c8.el10.x86_64 34/46 2026-02-19T13:08:28.610 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-rbd-2:20.2.0-720.g2fc421c8.el10.x86_64 35/46 2026-02-19T13:08:28.617 INFO:teuthology.orchestra.run.trial111.stdout: Installing : boost-program-options-1.83.0-5.el10.x86_64 36/46 2026-02-19T13:08:28.823 INFO:teuthology.orchestra.run.trial111.stdout: Installing : parquet-libs-17.0.0-9.el10_2.x86_64 37/46 2026-02-19T13:08:28.824 INFO:teuthology.orchestra.run.trial111.stdout: Installing : librgw2-2:20.2.0-720.g2fc421c8.el10.x86_64 38/46 2026-02-19T13:08:28.840 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: librgw2-2:20.2.0-720.g2fc421c8.el10.x86_64 38/46 2026-02-19T13:08:28.856 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-rgw-2:20.2.0-720.g2fc421c8.el10.x86_64 39/46 2026-02-19T13:08:28.858 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-ceph-common-2:20.2.0-720.g2fc421c8.el10.x8 40/46 2026-02-19T13:08:29.639 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-common-2:20.2.0-720.g2fc421c8.el10.x86_64 41/46 2026-02-19T13:08:29.641 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-common-2:20.2.0-720.g2fc421c8.el10.x86_64 41/46 2026-02-19T13:08:29.866 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-common-2:20.2.0-720.g2fc421c8.el10.x86_64 41/46 2026-02-19T13:08:29.869 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-base-2:20.2.0-720.g2fc421c8.el10.x86_64 42/46 2026-02-19T13:08:29.899 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-base-2:20.2.0-720.g2fc421c8.el10.x86_64 42/46 2026-02-19T13:08:29.899 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph.target' → '/usr/lib/systemd/system/ceph.target'. 2026-02-19T13:08:29.899 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-crash.service' → '/usr/lib/systemd/system/ceph-crash.service'. 2026-02-19T13:08:29.899 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:29.901 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-selinux-2:20.2.0-720.g2fc421c8.el10.x86_64 43/46 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-selinux-2:20.2.0-720.g2fc421c8.el10.x86_64 43/46 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout:skipping the directory /sys 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout:skipping the directory /proc 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout:skipping the directory /mnt 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout:skipping the directory /var/tmp 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout:skipping the directory /home 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout:skipping the directory /root 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout:skipping the directory /tmp 2026-02-19T13:08:33.800 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:34.559 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-radosgw-2:20.2.0-720.g2fc421c8.el10.x86_64 44/46 2026-02-19T13:08:34.583 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-radosgw-2:20.2.0-720.g2fc421c8.el10.x86_64 44/46 2026-02-19T13:08:34.583 INFO:teuthology.orchestra.run.trial111.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-19T13:08:34.583 INFO:teuthology.orchestra.run.trial111.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-19T13:08:34.583 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-radosgw.target' → '/usr/lib/systemd/system/ceph-radosgw.target'. 2026-02-19T13:08:34.583 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-radosgw.target' → '/usr/lib/systemd/system/ceph-radosgw.target'. 2026-02-19T13:08:34.583 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:34.586 INFO:teuthology.orchestra.run.trial111.stdout: Cleanup : librbd1-2:18.2.1-4.el10.x86_64 45/46 2026-02-19T13:08:34.605 INFO:teuthology.orchestra.run.trial111.stdout: Cleanup : librados2-2:18.2.1-4.el10.x86_64 46/46 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: librados2-2:18.2.1-4.el10.x86_64 46/46 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout:Upgraded: 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: librados2-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: librbd1-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: abseil-cpp-20240722.1-1.el10_0.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: boost-program-options-1.83.0-5.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: ceph-base-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: ceph-common-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: ceph-radosgw-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: ceph-selinux-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: daxctl-libs-80-4.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: gperftools-libs-2.15-4.el10_0.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libarrow-17.0.0-9.el10_2.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libarrow-doc-17.0.0-9.el10_2.noarch 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libcephfs-daemon-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libcephfs-proxy2-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libcephfs2-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libnbd-1.22.2-3.el10_1.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: liboath-2.6.12-1.el10_0.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: liborc2-2.0.7-1.el10_2.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libpmem-2.1.0-2.el10_0.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libpmemobj-2.1.0-2.el10_0.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: librabbitmq-0.14.0-3.el10.x86_64 2026-02-19T13:08:35.032 INFO:teuthology.orchestra.run.trial111.stdout: libradosstriper1-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: librdkafka-2.1.1-7.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: librgw2-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: libunwind-1.8.0-4.el10_0.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: lttng-ust-2.13.7-5.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: mailcap-2.1.54-8.el10.noarch 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: ndctl-libs-80-4.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: parquet-libs-17.0.0-9.el10_2.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: pciutils-3.13.0-6.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: protobuf-3.19.6-11.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: python3-ceph-argparse-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: python3-ceph-common-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: python3-cephfs-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: python3-prettytable-3.6.0-10.el10.noarch 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: python3-rados-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: python3-rbd-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: python3-rgw-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: qatlib-24.09.0-2.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: qatlib-service-24.09.0-2.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: qatzip-libs-1.3.1-1.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: re2-2:20251105-1.el10_2.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: thrift-0.20.0-4.el10_1.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: utf8proc-2.7.0-9.el10.x86_64 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:35.033 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:08:35.074 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-test 2026-02-19T13:08:35.365 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:17 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:08:35.402 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:08:35.403 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:35.403 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repository Size 2026-02-19T13:08:35.403 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout: ceph-test x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 86 M 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout:Installing dependencies: 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout: libcephsqlite x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 172 k 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout: libxslt x86_64 1.1.39-8.el10_0 appstream 189 k 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout: socat x86_64 1.7.4.4-8.el10 appstream 303 k 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout: xmlstarlet x86_64 1.6.1-27.el10 appstream 63 k 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:08:35.404 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:35.405 INFO:teuthology.orchestra.run.trial111.stdout:Install 5 Packages 2026-02-19T13:08:35.405 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:35.405 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 87 M 2026-02-19T13:08:35.405 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 349 M 2026-02-19T13:08:35.405 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:08:35.476 INFO:teuthology.orchestra.run.trial111.stdout:(1/5): libxslt-1.1.39-8.el10_0.x86_64.rpm 2.6 MB/s | 189 kB 00:00 2026-02-19T13:08:35.498 INFO:teuthology.orchestra.run.trial111.stdout:(2/5): socat-1.7.4.4-8.el10.x86_64.rpm 14 MB/s | 303 kB 00:00 2026-02-19T13:08:35.515 INFO:teuthology.orchestra.run.trial111.stdout:(3/5): xmlstarlet-1.6.1-27.el10.x86_64.rpm 3.6 MB/s | 63 kB 00:00 2026-02-19T13:08:35.892 INFO:teuthology.orchestra.run.trial111.stdout:(4/5): libcephsqlite-20.2.0-720.g2fc421c8.el10. 353 kB/s | 172 kB 00:00 2026-02-19T13:08:43.120 INFO:teuthology.orchestra.run.trial111.stdout:(5/5): ceph-test-20.2.0-720.g2fc421c8.el10.x86_ 11 MB/s | 86 MB 00:07 2026-02-19T13:08:43.121 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:08:43.122 INFO:teuthology.orchestra.run.trial111.stdout:Total 11 MB/s | 87 MB 00:07 2026-02-19T13:08:43.122 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:08:43.132 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:08:43.132 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:08:43.423 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:08:43.424 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:08:43.739 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:08:43.742 INFO:teuthology.orchestra.run.trial111.stdout: Installing : socat-1.7.4.4-8.el10.x86_64 1/5 2026-02-19T13:08:43.745 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libxslt-1.1.39-8.el10_0.x86_64 2/5 2026-02-19T13:08:43.751 INFO:teuthology.orchestra.run.trial111.stdout: Installing : xmlstarlet-1.6.1-27.el10.x86_64 3/5 2026-02-19T13:08:43.752 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libcephsqlite-2:20.2.0-720.g2fc421c8.el10.x86_64 4/5 2026-02-19T13:08:46.508 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: libcephsqlite-2:20.2.0-720.g2fc421c8.el10.x86_64 4/5 2026-02-19T13:08:46.528 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-test-2:20.2.0-720.g2fc421c8.el10.x86_64 5/5 2026-02-19T13:08:46.892 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-test-2:20.2.0-720.g2fc421c8.el10.x86_64 5/5 2026-02-19T13:08:46.892 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:46.892 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:08:46.892 INFO:teuthology.orchestra.run.trial111.stdout: ceph-test-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:46.892 INFO:teuthology.orchestra.run.trial111.stdout: libcephsqlite-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:46.892 INFO:teuthology.orchestra.run.trial111.stdout: libxslt-1.1.39-8.el10_0.x86_64 2026-02-19T13:08:46.892 INFO:teuthology.orchestra.run.trial111.stdout: socat-1.7.4.4-8.el10.x86_64 2026-02-19T13:08:46.893 INFO:teuthology.orchestra.run.trial111.stdout: xmlstarlet-1.6.1-27.el10.x86_64 2026-02-19T13:08:46.893 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:46.893 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:08:46.931 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph 2026-02-19T13:08:47.225 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:29 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:08:47.264 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:08:47.266 INFO:teuthology.orchestra.run.trial111.stdout:======================================================================================= 2026-02-19T13:08:47.266 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repository Size 2026-02-19T13:08:47.266 INFO:teuthology.orchestra.run.trial111.stdout:======================================================================================= 2026-02-19T13:08:47.266 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:08:47.266 INFO:teuthology.orchestra.run.trial111.stdout: ceph x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 6.6 k 2026-02-19T13:08:47.266 INFO:teuthology.orchestra.run.trial111.stdout:Installing dependencies: 2026-02-19T13:08:47.266 INFO:teuthology.orchestra.run.trial111.stdout: ceph-grafana-dashboards noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 43 k 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mds x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 2.4 M 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 997 k 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-modules-core noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 296 k 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mon x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 5.2 M 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: ceph-osd x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 18 M 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: ceph-prometheus-alerts noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 17 k 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: cephadm noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 1.3 M 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: cryptsetup x86_64 2.7.5-2.el10 baseos 335 k 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: flexiblas x86_64 3.4.2-3.el10 appstream 25 k 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: flexiblas-netlib x86_64 3.4.2-3.el10 appstream 3.2 M 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: flexiblas-openblas-openmp x86_64 3.4.2-3.el10 appstream 17 k 2026-02-19T13:08:47.267 INFO:teuthology.orchestra.run.trial111.stdout: grpc-data noarch 1.48.4-44.el10_0 epel 21 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: ledmon-libs x86_64 1.1.0-2.el10 appstream 42 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: libconfig x86_64 1.7.3-10.el10 baseos 72 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: libgfortran x86_64 14.3.1-2.1.el10 baseos 827 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: libquadmath x86_64 14.3.1-2.1.el10 baseos 215 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: libstoragemgmt x86_64 1.10.1-4.el10 appstream 251 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: libtool-ltdl x86_64 2.4.7-13.el10 appstream 36 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: lua x86_64 5.4.6-7.el10 appstream 190 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: lua-devel x86_64 5.4.6-7.el10 crb 22 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: luarocks noarch 3.9.2-6.el10_0 epel 152 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: openblas x86_64 0.3.29-2.el10 appstream 43 k 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: openblas-openmp x86_64 0.3.29-2.el10 appstream 5.3 M 2026-02-19T13:08:47.268 INFO:teuthology.orchestra.run.trial111.stdout: protobuf-compiler x86_64 3.19.6-11.el10 crb 787 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-asyncssh noarch 2.17.0-1.el10_0 epel 756 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-autocommand noarch 2.2.2-9.el10_0 epel 32 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-bcrypt x86_64 4.3.0-3.el10_1 epel 238 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-cachetools noarch 5.4.0-6.el10_0 epel 37 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-certifi noarch 2023.05.07-6.el10_0 epel 14 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-cffi x86_64 1.16.0-7.el10 baseos 310 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-cheroot noarch 10.0.1-4.el10_0 epel 215 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-cherrypy noarch 18.10.0-4.el10_0 epel 359 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-cryptography x86_64 43.0.0-4.el10 baseos 1.3 M 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-devel x86_64 3.12.12-3.el10_1 appstream 329 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-durationpy noarch 0.9-2.el10_0 epel 15 k 2026-02-19T13:08:47.269 INFO:teuthology.orchestra.run.trial111.stdout: python3-google-auth noarch 1:2.40.3-1.el10_1 epel 286 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-grpcio x86_64 1.48.4-44.el10_0 epel 2.3 M 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-grpcio-tools x86_64 1.48.4-44.el10_0 epel 157 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-isodate noarch 0.6.1-14.el10_0 epel 66 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-jaraco-collections noarch 5.1.0-1.el10_0 epel 29 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-jaraco-context noarch 6.0.1-4.el10_0 epel 21 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-jaraco-functools noarch 4.0.2-4.el10_0 epel 23 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-jaraco-text noarch 4.0.0-3.el10_0 epel 29 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-jinja2 noarch 3.1.6-1.el10_0 appstream 330 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-jmespath noarch 1.0.1-8.el10 appstream 60 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-kubernetes noarch 1:32.0.1-1.el10_1 epel 1.3 M 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-libstoragemgmt x86_64 1.10.1-4.el10 appstream 219 k 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-lxml x86_64 5.2.1-4.el10 appstream 1.4 M 2026-02-19T13:08:47.270 INFO:teuthology.orchestra.run.trial111.stdout: python3-markupsafe x86_64 2.1.3-6.el10 appstream 34 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-more-itertools noarch 10.1.0-8.el10_0 epel 100 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-natsort noarch 8.4.0-2.el10_0 epel 72 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-numpy x86_64 1:1.26.4-5.el10 appstream 6.7 M 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-numpy-f2py x86_64 1:1.26.4-5.el10 appstream 490 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-packaging noarch 24.2-2.el10 baseos 155 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-ply noarch 3.11-25.el10 baseos 137 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-portend noarch 3.2.0-6.el10_0 epel 18 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-protobuf x86_64 3.19.6-11.el10 appstream 419 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-pyOpenSSL noarch 25.0.0-1.el10_1 epel 113 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-pyasn1 noarch 0.6.1-1.el10 appstream 198 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-pyasn1-modules noarch 0.6.1-1.el10 appstream 402 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-pycparser noarch 2.20-16.el10 baseos 160 k 2026-02-19T13:08:47.271 INFO:teuthology.orchestra.run.trial111.stdout: python3-repoze-lru noarch 0.7-23.el10_0 epel 36 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-requests noarch 2.32.4-1.el10_0 baseos 160 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-requests-oauthlib noarch 1.3.1-11.el10 appstream 60 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-routes noarch 2.5.1-11.el10_0 epel 201 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-rsa noarch 4.9-7.el10_0 epel 70 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-scipy x86_64 1.11.3-17.el10 appstream 16 M 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-tempora noarch 5.7.0-1.el10_0 epel 37 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-urllib3 noarch 1.26.19-2.el10_1.1 baseos 285 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-websocket-client noarch 1.8.0-2.el10_0 epel 124 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-xmlsec x86_64 1.3.16-1.el10 lab-extras 49 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-xmltodict noarch 0.13.0-5.el10_0 epel 26 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: python3-zc-lockfile noarch 3.0.post1-11.el10_0 epel 23 k 2026-02-19T13:08:47.272 INFO:teuthology.orchestra.run.trial111.stdout: unzip x86_64 6.0-69.el10 baseos 188 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: xmlsec1 x86_64 1:1.2.39-3.el10 appstream 193 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: xmlsec1-openssl x86_64 1:1.2.39-3.el10 appstream 96 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: zip x86_64 3.0-45.el10 baseos 271 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout:Installing weak dependencies: 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-cephadm noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 174 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-dashboard noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 11 M 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-diskprediction-local noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 7.4 M 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-k8sevents noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 22 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-rook noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 50 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: ceph-volume noarch 2:20.2.0-720.g2fc421c8.el10 ceph-noarch 371 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: python3-saml noarch 1.16.0-11.el10 lab-extras 150 k 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:47.273 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:08:47.274 INFO:teuthology.orchestra.run.trial111.stdout:======================================================================================= 2026-02-19T13:08:47.274 INFO:teuthology.orchestra.run.trial111.stdout:Install 86 Packages 2026-02-19T13:08:47.274 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:47.274 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 95 M 2026-02-19T13:08:47.275 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 535 M 2026-02-19T13:08:47.275 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:08:47.479 INFO:teuthology.orchestra.run.trial111.stdout:(1/86): ceph-20.2.0-720.g2fc421c8.el10.x86_64.r 52 kB/s | 6.6 kB 00:00 2026-02-19T13:08:48.013 INFO:teuthology.orchestra.run.trial111.stdout:(2/86): ceph-mgr-20.2.0-720.g2fc421c8.el10.x86_ 1.5 MB/s | 997 kB 00:00 2026-02-19T13:08:49.012 INFO:teuthology.orchestra.run.trial111.stdout:(3/86): ceph-mds-20.2.0-720.g2fc421c8.el10.x86_ 1.4 MB/s | 2.4 MB 00:01 2026-02-19T13:08:49.075 INFO:teuthology.orchestra.run.trial111.stdout:(4/86): ceph-grafana-dashboards-20.2.0-720.g2fc 685 kB/s | 43 kB 00:00 2026-02-19T13:08:49.203 INFO:teuthology.orchestra.run.trial111.stdout:(5/86): ceph-mgr-cephadm-20.2.0-720.g2fc421c8.e 1.3 MB/s | 174 kB 00:00 2026-02-19T13:08:50.788 INFO:teuthology.orchestra.run.trial111.stdout:(6/86): ceph-osd-20.2.0-720.g2fc421c8.el10.x86_ 6.3 MB/s | 18 MB 00:02 2026-02-19T13:08:51.038 INFO:teuthology.orchestra.run.trial111.stdout:(7/86): ceph-mon-20.2.0-720.g2fc421c8.el10.x86_ 1.5 MB/s | 5.2 MB 00:03 2026-02-19T13:08:51.106 INFO:teuthology.orchestra.run.trial111.stdout:(8/86): ceph-mgr-k8sevents-20.2.0-720.g2fc421c8 331 kB/s | 22 kB 00:00 2026-02-19T13:08:51.296 INFO:teuthology.orchestra.run.trial111.stdout:(9/86): ceph-mgr-modules-core-20.2.0-720.g2fc42 1.5 MB/s | 296 kB 00:00 2026-02-19T13:08:51.306 INFO:teuthology.orchestra.run.trial111.stdout:(10/86): ceph-mgr-diskprediction-local-20.2.0-7 14 MB/s | 7.4 MB 00:00 2026-02-19T13:08:51.337 INFO:teuthology.orchestra.run.trial111.stdout:(11/86): ceph-mgr-rook-20.2.0-720.g2fc421c8.el1 1.2 MB/s | 50 kB 00:00 2026-02-19T13:08:51.347 INFO:teuthology.orchestra.run.trial111.stdout:(12/86): ceph-prometheus-alerts-20.2.0-720.g2fc 424 kB/s | 17 kB 00:00 2026-02-19T13:08:51.455 INFO:teuthology.orchestra.run.trial111.stdout:(13/86): cephadm-20.2.0-720.g2fc421c8.el10.noar 12 MB/s | 1.3 MB 00:00 2026-02-19T13:08:51.486 INFO:teuthology.orchestra.run.trial111.stdout:(14/86): flexiblas-3.4.2-3.el10.x86_64.rpm 792 kB/s | 25 kB 00:00 2026-02-19T13:08:51.528 INFO:teuthology.orchestra.run.trial111.stdout:(15/86): ceph-volume-20.2.0-720.g2fc421c8.el10. 1.9 MB/s | 371 kB 00:00 2026-02-19T13:08:51.538 INFO:teuthology.orchestra.run.trial111.stdout:(16/86): flexiblas-netlib-3.4.2-3.el10.x86_64.r 62 MB/s | 3.2 MB 00:00 2026-02-19T13:08:51.540 INFO:teuthology.orchestra.run.trial111.stdout:(17/86): flexiblas-openblas-openmp-3.4.2-3.el10 1.4 MB/s | 17 kB 00:00 2026-02-19T13:08:51.552 INFO:teuthology.orchestra.run.trial111.stdout:(18/86): ledmon-libs-1.1.0-2.el10.x86_64.rpm 2.9 MB/s | 42 kB 00:00 2026-02-19T13:08:51.558 INFO:teuthology.orchestra.run.trial111.stdout:(19/86): libstoragemgmt-1.10.1-4.el10.x86_64.rp 14 MB/s | 251 kB 00:00 2026-02-19T13:08:51.568 INFO:teuthology.orchestra.run.trial111.stdout:(20/86): libtool-ltdl-2.4.7-13.el10.x86_64.rpm 2.2 MB/s | 36 kB 00:00 2026-02-19T13:08:51.575 INFO:teuthology.orchestra.run.trial111.stdout:(21/86): lua-5.4.6-7.el10.x86_64.rpm 11 MB/s | 190 kB 00:00 2026-02-19T13:08:51.584 INFO:teuthology.orchestra.run.trial111.stdout:(22/86): openblas-0.3.29-2.el10.x86_64.rpm 2.7 MB/s | 43 kB 00:00 2026-02-19T13:08:51.605 INFO:teuthology.orchestra.run.trial111.stdout:(23/86): python3-devel-3.12.12-3.el10_1.x86_64. 15 MB/s | 329 kB 00:00 2026-02-19T13:08:51.634 INFO:teuthology.orchestra.run.trial111.stdout:(24/86): python3-jinja2-3.1.6-1.el10_0.noarch.r 11 MB/s | 330 kB 00:00 2026-02-19T13:08:51.650 INFO:teuthology.orchestra.run.trial111.stdout:(25/86): python3-jmespath-1.0.1-8.el10.noarch.r 3.8 MB/s | 60 kB 00:00 2026-02-19T13:08:51.661 INFO:teuthology.orchestra.run.trial111.stdout:(26/86): openblas-openmp-0.3.29-2.el10.x86_64.r 62 MB/s | 5.3 MB 00:00 2026-02-19T13:08:51.672 INFO:teuthology.orchestra.run.trial111.stdout:(27/86): python3-libstoragemgmt-1.10.1-4.el10.x 9.8 MB/s | 219 kB 00:00 2026-02-19T13:08:51.688 INFO:teuthology.orchestra.run.trial111.stdout:(28/86): python3-markupsafe-2.1.3-6.el10.x86_64 2.1 MB/s | 34 kB 00:00 2026-02-19T13:08:51.693 INFO:teuthology.orchestra.run.trial111.stdout:(29/86): python3-lxml-5.2.1-4.el10.x86_64.rpm 43 MB/s | 1.4 MB 00:00 2026-02-19T13:08:51.719 INFO:teuthology.orchestra.run.trial111.stdout:(30/86): python3-numpy-f2py-1.26.4-5.el10.x86_6 19 MB/s | 490 kB 00:00 2026-02-19T13:08:51.742 INFO:teuthology.orchestra.run.trial111.stdout:(31/86): python3-protobuf-3.19.6-11.el10.x86_64 18 MB/s | 419 kB 00:00 2026-02-19T13:08:51.772 INFO:teuthology.orchestra.run.trial111.stdout:(32/86): python3-pyasn1-0.6.1-1.el10.noarch.rpm 6.6 MB/s | 198 kB 00:00 2026-02-19T13:08:51.801 INFO:teuthology.orchestra.run.trial111.stdout:(33/86): python3-pyasn1-modules-0.6.1-1.el10.no 14 MB/s | 402 kB 00:00 2026-02-19T13:08:51.809 INFO:teuthology.orchestra.run.trial111.stdout:(34/86): python3-numpy-1.26.4-5.el10.x86_64.rpm 55 MB/s | 6.7 MB 00:00 2026-02-19T13:08:51.814 INFO:teuthology.orchestra.run.trial111.stdout:(35/86): python3-requests-oauthlib-1.3.1-11.el1 4.5 MB/s | 60 kB 00:00 2026-02-19T13:08:51.830 INFO:teuthology.orchestra.run.trial111.stdout:(36/86): xmlsec1-1.2.39-3.el10.x86_64.rpm 12 MB/s | 193 kB 00:00 2026-02-19T13:08:51.847 INFO:teuthology.orchestra.run.trial111.stdout:(37/86): xmlsec1-openssl-1.2.39-3.el10.x86_64.r 5.5 MB/s | 96 kB 00:00 2026-02-19T13:08:51.866 INFO:teuthology.orchestra.run.trial111.stdout:(38/86): cryptsetup-2.7.5-2.el10.x86_64.rpm 17 MB/s | 335 kB 00:00 2026-02-19T13:08:51.883 INFO:teuthology.orchestra.run.trial111.stdout:(39/86): libconfig-1.7.3-10.el10.x86_64.rpm 4.2 MB/s | 72 kB 00:00 2026-02-19T13:08:51.907 INFO:teuthology.orchestra.run.trial111.stdout:(40/86): libgfortran-14.3.1-2.1.el10.x86_64.rpm 34 MB/s | 827 kB 00:00 2026-02-19T13:08:51.925 INFO:teuthology.orchestra.run.trial111.stdout:(41/86): libquadmath-14.3.1-2.1.el10.x86_64.rpm 12 MB/s | 215 kB 00:00 2026-02-19T13:08:51.943 INFO:teuthology.orchestra.run.trial111.stdout:(42/86): python3-cffi-1.16.0-7.el10.x86_64.rpm 17 MB/s | 310 kB 00:00 2026-02-19T13:08:51.973 INFO:teuthology.orchestra.run.trial111.stdout:(43/86): python3-cryptography-43.0.0-4.el10.x86 45 MB/s | 1.3 MB 00:00 2026-02-19T13:08:51.991 INFO:teuthology.orchestra.run.trial111.stdout:(44/86): python3-packaging-24.2-2.el10.noarch.r 8.3 MB/s | 155 kB 00:00 2026-02-19T13:08:52.015 INFO:teuthology.orchestra.run.trial111.stdout:(45/86): python3-scipy-1.11.3-17.el10.x86_64.rp 79 MB/s | 16 MB 00:00 2026-02-19T13:08:52.019 INFO:teuthology.orchestra.run.trial111.stdout:(46/86): python3-ply-3.11-25.el10.noarch.rpm 4.8 MB/s | 137 kB 00:00 2026-02-19T13:08:52.031 INFO:teuthology.orchestra.run.trial111.stdout:(47/86): python3-pycparser-2.20-16.el10.noarch. 9.8 MB/s | 160 kB 00:00 2026-02-19T13:08:52.035 INFO:teuthology.orchestra.run.trial111.stdout:(48/86): python3-requests-2.32.4-1.el10_0.noarc 9.3 MB/s | 160 kB 00:00 2026-02-19T13:08:52.050 INFO:teuthology.orchestra.run.trial111.stdout:(49/86): python3-urllib3-1.26.19-2.el10_1.1.noa 14 MB/s | 285 kB 00:00 2026-02-19T13:08:52.052 INFO:teuthology.orchestra.run.trial111.stdout:(50/86): unzip-6.0-69.el10.x86_64.rpm 11 MB/s | 188 kB 00:00 2026-02-19T13:08:52.065 INFO:teuthology.orchestra.run.trial111.stdout:(51/86): lua-devel-5.4.6-7.el10.x86_64.rpm 1.7 MB/s | 22 kB 00:00 2026-02-19T13:08:52.069 INFO:teuthology.orchestra.run.trial111.stdout:(52/86): zip-3.0-45.el10.x86_64.rpm 14 MB/s | 271 kB 00:00 2026-02-19T13:08:52.085 INFO:teuthology.orchestra.run.trial111.stdout:(53/86): protobuf-compiler-3.19.6-11.el10.x86_6 38 MB/s | 787 kB 00:00 2026-02-19T13:08:52.170 INFO:teuthology.orchestra.run.trial111.stdout:(54/86): grpc-data-1.48.4-44.el10_0.noarch.rpm 209 kB/s | 21 kB 00:00 2026-02-19T13:08:52.230 INFO:teuthology.orchestra.run.trial111.stdout:(55/86): luarocks-3.9.2-6.el10_0.noarch.rpm 1.0 MB/s | 152 kB 00:00 2026-02-19T13:08:52.254 INFO:teuthology.orchestra.run.trial111.stdout:(56/86): python3-autocommand-2.2.2-9.el10_0.noa 1.3 MB/s | 32 kB 00:00 2026-02-19T13:08:52.297 INFO:teuthology.orchestra.run.trial111.stdout:(57/86): python3-asyncssh-2.17.0-1.el10_0.noarc 5.8 MB/s | 756 kB 00:00 2026-02-19T13:08:52.300 INFO:teuthology.orchestra.run.trial111.stdout:(58/86): python3-bcrypt-4.3.0-3.el10_1.x86_64.r 5.0 MB/s | 238 kB 00:00 2026-02-19T13:08:52.320 INFO:teuthology.orchestra.run.trial111.stdout:(59/86): python3-cachetools-5.4.0-6.el10_0.noar 1.6 MB/s | 37 kB 00:00 2026-02-19T13:08:52.323 INFO:teuthology.orchestra.run.trial111.stdout:(60/86): python3-certifi-2023.05.07-6.el10_0.no 650 kB/s | 14 kB 00:00 2026-02-19T13:08:52.347 INFO:teuthology.orchestra.run.trial111.stdout:(61/86): python3-cheroot-10.0.1-4.el10_0.noarch 7.8 MB/s | 215 kB 00:00 2026-02-19T13:08:52.358 INFO:teuthology.orchestra.run.trial111.stdout:(62/86): python3-cherrypy-18.10.0-4.el10_0.noar 10 MB/s | 359 kB 00:00 2026-02-19T13:08:52.369 INFO:teuthology.orchestra.run.trial111.stdout:(63/86): python3-durationpy-0.9-2.el10_0.noarch 647 kB/s | 15 kB 00:00 2026-02-19T13:08:52.388 INFO:teuthology.orchestra.run.trial111.stdout:(64/86): python3-google-auth-2.40.3-1.el10_1.no 9.0 MB/s | 286 kB 00:00 2026-02-19T13:08:52.415 INFO:teuthology.orchestra.run.trial111.stdout:(65/86): python3-grpcio-tools-1.48.4-44.el10_0. 5.8 MB/s | 157 kB 00:00 2026-02-19T13:08:52.439 INFO:teuthology.orchestra.run.trial111.stdout:(66/86): python3-grpcio-1.48.4-44.el10_0.x86_64 33 MB/s | 2.3 MB 00:00 2026-02-19T13:08:52.439 INFO:teuthology.orchestra.run.trial111.stdout:(67/86): python3-isodate-0.6.1-14.el10_0.noarch 2.7 MB/s | 66 kB 00:00 2026-02-19T13:08:52.462 INFO:teuthology.orchestra.run.trial111.stdout:(68/86): python3-jaraco-collections-5.1.0-1.el1 1.3 MB/s | 29 kB 00:00 2026-02-19T13:08:52.462 INFO:teuthology.orchestra.run.trial111.stdout:(69/86): python3-jaraco-context-6.0.1-4.el10_0. 943 kB/s | 21 kB 00:00 2026-02-19T13:08:52.484 INFO:teuthology.orchestra.run.trial111.stdout:(70/86): python3-jaraco-functools-4.0.2-4.el10_ 1.0 MB/s | 23 kB 00:00 2026-02-19T13:08:52.485 INFO:teuthology.orchestra.run.trial111.stdout:(71/86): python3-jaraco-text-4.0.0-3.el10_0.noa 1.3 MB/s | 29 kB 00:00 2026-02-19T13:08:52.509 INFO:teuthology.orchestra.run.trial111.stdout:(72/86): python3-more-itertools-10.1.0-8.el10_0 4.0 MB/s | 100 kB 00:00 2026-02-19T13:08:52.516 INFO:teuthology.orchestra.run.trial111.stdout:(73/86): python3-kubernetes-32.0.1-1.el10_1.noa 42 MB/s | 1.3 MB 00:00 2026-02-19T13:08:52.533 INFO:teuthology.orchestra.run.trial111.stdout:(74/86): python3-natsort-8.4.0-2.el10_0.noarch. 3.0 MB/s | 72 kB 00:00 2026-02-19T13:08:52.539 INFO:teuthology.orchestra.run.trial111.stdout:(75/86): python3-portend-3.2.0-6.el10_0.noarch. 776 kB/s | 18 kB 00:00 2026-02-19T13:08:52.559 INFO:teuthology.orchestra.run.trial111.stdout:(76/86): python3-pyOpenSSL-25.0.0-1.el10_1.noar 4.3 MB/s | 113 kB 00:00 2026-02-19T13:08:52.562 INFO:teuthology.orchestra.run.trial111.stdout:(77/86): python3-repoze-lru-0.7-23.el10_0.noarc 1.6 MB/s | 36 kB 00:00 2026-02-19T13:08:52.584 INFO:teuthology.orchestra.run.trial111.stdout:(78/86): python3-rsa-4.9-7.el10_0.noarch.rpm 3.0 MB/s | 70 kB 00:00 2026-02-19T13:08:52.586 INFO:teuthology.orchestra.run.trial111.stdout:(79/86): python3-routes-2.5.1-11.el10_0.noarch. 7.1 MB/s | 201 kB 00:00 2026-02-19T13:08:52.607 INFO:teuthology.orchestra.run.trial111.stdout:(80/86): python3-tempora-5.7.0-1.el10_0.noarch. 1.6 MB/s | 37 kB 00:00 2026-02-19T13:08:52.612 INFO:teuthology.orchestra.run.trial111.stdout:(81/86): python3-websocket-client-1.8.0-2.el10_ 4.8 MB/s | 124 kB 00:00 2026-02-19T13:08:52.630 INFO:teuthology.orchestra.run.trial111.stdout:(82/86): python3-xmltodict-0.13.0-5.el10_0.noar 1.1 MB/s | 26 kB 00:00 2026-02-19T13:08:52.634 INFO:teuthology.orchestra.run.trial111.stdout:(83/86): python3-zc-lockfile-3.0.post1-11.el10_ 1.0 MB/s | 23 kB 00:00 2026-02-19T13:08:52.635 INFO:teuthology.orchestra.run.trial111.stdout:(84/86): python3-saml-1.16.0-11.el10.noarch.rpm 28 MB/s | 150 kB 00:00 2026-02-19T13:08:52.636 INFO:teuthology.orchestra.run.trial111.stdout:(85/86): python3-xmlsec-1.3.16-1.el10.x86_64.rp 30 MB/s | 49 kB 00:00 2026-02-19T13:08:54.227 INFO:teuthology.orchestra.run.trial111.stdout:(86/86): ceph-mgr-dashboard-20.2.0-720.g2fc421c 2.1 MB/s | 11 MB 00:05 2026-02-19T13:08:54.228 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:08:54.229 INFO:teuthology.orchestra.run.trial111.stdout:Total 14 MB/s | 95 MB 00:06 2026-02-19T13:08:54.229 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:08:54.269 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:08:54.269 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:08:54.654 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:08:54.654 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:08:55.060 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:08:55.062 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-more-itertools-10.1.0-8.el10_0.noarch 1/86 2026-02-19T13:08:55.069 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-jaraco-functools-4.0.2-4.el10_0.noarch 2/86 2026-02-19T13:08:55.074 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libgfortran-14.3.1-2.1.el10.x86_64 3/86 2026-02-19T13:08:55.084 INFO:teuthology.orchestra.run.trial111.stdout: Installing : unzip-6.0-69.el10.x86_64 4/86 2026-02-19T13:08:55.089 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-urllib3-1.26.19-2.el10_1.1.noarch 5/86 2026-02-19T13:08:55.094 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-requests-2.32.4-1.el10_0.noarch 6/86 2026-02-19T13:08:55.103 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-packaging-24.2-2.el10.noarch 7/86 2026-02-19T13:08:55.122 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-pyasn1-0.6.1-1.el10.noarch 8/86 2026-02-19T13:08:55.127 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-lxml-5.2.1-4.el10.x86_64 9/86 2026-02-19T13:08:55.128 INFO:teuthology.orchestra.run.trial111.stdout: Installing : lua-5.4.6-7.el10.x86_64 10/86 2026-02-19T13:08:55.131 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libtool-ltdl-2.4.7-13.el10.x86_64 11/86 2026-02-19T13:08:55.132 INFO:teuthology.orchestra.run.trial111.stdout: Installing : xmlsec1-1:1.2.39-3.el10.x86_64 12/86 2026-02-19T13:08:55.134 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ledmon-libs-1.1.0-2.el10.x86_64 13/86 2026-02-19T13:08:55.136 INFO:teuthology.orchestra.run.trial111.stdout: Installing : xmlsec1-openssl-1:1.2.39-3.el10.x86_64 14/86 2026-02-19T13:08:55.138 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-xmlsec-1.3.16-1.el10.x86_64 15/86 2026-02-19T13:08:55.160 INFO:teuthology.orchestra.run.trial111.stdout: Installing : lua-devel-5.4.6-7.el10.x86_64 16/86 2026-02-19T13:08:55.165 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-pyasn1-modules-0.6.1-1.el10.noarch 17/86 2026-02-19T13:08:55.170 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-rsa-4.9-7.el10_0.noarch 18/86 2026-02-19T13:08:55.174 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-requests-oauthlib-1.3.1-11.el10.noarch 19/86 2026-02-19T13:08:55.183 INFO:teuthology.orchestra.run.trial111.stdout: Installing : zip-3.0-45.el10.x86_64 20/86 2026-02-19T13:08:55.187 INFO:teuthology.orchestra.run.trial111.stdout: Installing : luarocks-3.9.2-6.el10_0.noarch 21/86 2026-02-19T13:08:55.189 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-tempora-5.7.0-1.el10_0.noarch 22/86 2026-02-19T13:08:55.191 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-portend-3.2.0-6.el10_0.noarch 23/86 2026-02-19T13:08:55.193 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-zc-lockfile-3.0.post1-11.el10_0.noarch 24/86 2026-02-19T13:08:55.198 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-xmltodict-0.13.0-5.el10_0.noarch 25/86 2026-02-19T13:08:55.201 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-websocket-client-1.8.0-2.el10_0.noarch 26/86 2026-02-19T13:08:55.206 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-repoze-lru-0.7-23.el10_0.noarch 27/86 2026-02-19T13:08:55.210 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-routes-2.5.1-11.el10_0.noarch 28/86 2026-02-19T13:08:55.213 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-natsort-8.4.0-2.el10_0.noarch 29/86 2026-02-19T13:08:55.217 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-jaraco-context-6.0.1-4.el10_0.noarch 30/86 2026-02-19T13:08:55.223 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-isodate-0.6.1-14.el10_0.noarch 31/86 2026-02-19T13:08:55.225 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-saml-1.16.0-11.el10.noarch 32/86 2026-02-19T13:08:55.227 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-durationpy-0.9-2.el10_0.noarch 33/86 2026-02-19T13:08:55.230 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-certifi-2023.05.07-6.el10_0.noarch 34/86 2026-02-19T13:08:55.244 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-cachetools-5.4.0-6.el10_0.noarch 35/86 2026-02-19T13:08:55.380 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-google-auth-1:2.40.3-1.el10_1.noarch 36/86 2026-02-19T13:08:55.388 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-kubernetes-1:32.0.1-1.el10_1.noarch 37/86 2026-02-19T13:08:55.391 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-bcrypt-4.3.0-3.el10_1.x86_64 38/86 2026-02-19T13:08:55.394 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-autocommand-2.2.2-9.el10_0.noarch 39/86 2026-02-19T13:08:55.396 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-jaraco-text-4.0.0-3.el10_0.noarch 40/86 2026-02-19T13:08:55.398 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-jaraco-collections-5.1.0-1.el10_0.noarch 41/86 2026-02-19T13:08:55.420 INFO:teuthology.orchestra.run.trial111.stdout: Installing : grpc-data-1.48.4-44.el10_0.noarch 42/86 2026-02-19T13:08:55.428 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-grpcio-1.48.4-44.el10_0.x86_64 43/86 2026-02-19T13:08:55.432 INFO:teuthology.orchestra.run.trial111.stdout: Installing : protobuf-compiler-3.19.6-11.el10.x86_64 44/86 2026-02-19T13:08:55.437 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-ply-3.11-25.el10.noarch 45/86 2026-02-19T13:08:55.444 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-pycparser-2.20-16.el10.noarch 46/86 2026-02-19T13:08:55.476 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-cffi-1.16.0-7.el10.x86_64 47/86 2026-02-19T13:08:55.481 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-cryptography-43.0.0-4.el10.x86_64 48/86 2026-02-19T13:08:55.490 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-pyOpenSSL-25.0.0-1.el10_1.noarch 49/86 2026-02-19T13:08:55.502 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-cheroot-10.0.1-4.el10_0.noarch 50/86 2026-02-19T13:08:55.522 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-cherrypy-18.10.0-4.el10_0.noarch 51/86 2026-02-19T13:08:55.525 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-asyncssh-2.17.0-1.el10_0.noarch 52/86 2026-02-19T13:08:55.528 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libquadmath-14.3.1-2.1.el10.x86_64 53/86 2026-02-19T13:08:55.529 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libconfig-1.7.3-10.el10.x86_64 54/86 2026-02-19T13:08:55.549 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: libstoragemgmt-1.10.1-4.el10.x86_64 55/86 2026-02-19T13:08:55.549 INFO:teuthology.orchestra.run.trial111.stdout:Creating group 'libstoragemgmt' with GID 992. 2026-02-19T13:08:55.549 INFO:teuthology.orchestra.run.trial111.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 992 and GID 992. 2026-02-19T13:08:55.549 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:55.555 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libstoragemgmt-1.10.1-4.el10.x86_64 55/86 2026-02-19T13:08:55.578 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: libstoragemgmt-1.10.1-4.el10.x86_64 55/86 2026-02-19T13:08:55.578 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/libstoragemgmt.service' → '/usr/lib/systemd/system/libstoragemgmt.service'. 2026-02-19T13:08:55.579 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:55.594 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-libstoragemgmt-1.10.1-4.el10.x86_64 56/86 2026-02-19T13:08:55.624 INFO:teuthology.orchestra.run.trial111.stdout: Installing : cryptsetup-2.7.5-2.el10.x86_64 57/86 2026-02-19T13:08:55.626 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-volume-2:20.2.0-720.g2fc421c8.el10.noarch 58/86 2026-02-19T13:08:55.641 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-volume-2:20.2.0-720.g2fc421c8.el10.noarch 58/86 2026-02-19T13:08:55.641 INFO:teuthology.orchestra.run.trial111.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-19T13:08:55.641 INFO:teuthology.orchestra.run.trial111.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-19T13:08:55.641 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:56.153 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-osd-2:20.2.0-720.g2fc421c8.el10.x86_64 59/86 2026-02-19T13:08:56.178 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-osd-2:20.2.0-720.g2fc421c8.el10.x86_64 59/86 2026-02-19T13:08:56.178 INFO:teuthology.orchestra.run.trial111.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-19T13:08:56.179 INFO:teuthology.orchestra.run.trial111.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-19T13:08:56.179 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-osd.target' → '/usr/lib/systemd/system/ceph-osd.target'. 2026-02-19T13:08:56.179 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-osd.target' → '/usr/lib/systemd/system/ceph-osd.target'. 2026-02-19T13:08:56.179 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:56.198 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-protobuf-3.19.6-11.el10.x86_64 60/86 2026-02-19T13:08:56.201 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-grpcio-tools-1.48.4-44.el10_0.x86_64 61/86 2026-02-19T13:08:56.209 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-markupsafe-2.1.3-6.el10.x86_64 62/86 2026-02-19T13:08:56.213 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-jinja2-3.1.6-1.el10_0.noarch 63/86 2026-02-19T13:08:56.227 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-jmespath-1.0.1-8.el10.noarch 64/86 2026-02-19T13:08:56.230 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-devel-3.12.12-3.el10_1.x86_64 65/86 2026-02-19T13:08:56.285 INFO:teuthology.orchestra.run.trial111.stdout: Installing : openblas-0.3.29-2.el10.x86_64 66/86 2026-02-19T13:08:56.305 INFO:teuthology.orchestra.run.trial111.stdout: Installing : openblas-openmp-0.3.29-2.el10.x86_64 67/86 2026-02-19T13:08:56.306 INFO:teuthology.orchestra.run.trial111.stdout: Installing : flexiblas-netlib-3.4.2-3.el10.x86_64 68/86 2026-02-19T13:08:56.307 INFO:teuthology.orchestra.run.trial111.stdout: Installing : flexiblas-openblas-openmp-3.4.2-3.el10.x86_64 69/86 2026-02-19T13:08:56.432 INFO:teuthology.orchestra.run.trial111.stdout: Installing : flexiblas-3.4.2-3.el10.x86_64 70/86 2026-02-19T13:08:56.459 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-numpy-1:1.26.4-5.el10.x86_64 71/86 2026-02-19T13:08:56.645 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-numpy-f2py-1:1.26.4-5.el10.x86_64 72/86 2026-02-19T13:08:56.653 INFO:teuthology.orchestra.run.trial111.stdout: Installing : python3-scipy-1.11.3-17.el10.x86_64 73/86 2026-02-19T13:08:56.688 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: cephadm-2:20.2.0-720.g2fc421c8.el10.noarch 74/86 2026-02-19T13:08:56.690 INFO:teuthology.orchestra.run.trial111.stdout: Installing : cephadm-2:20.2.0-720.g2fc421c8.el10.noarch 74/86 2026-02-19T13:08:56.695 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-prometheus-alerts-2:20.2.0-720.g2fc421c8.el10 75/86 2026-02-19T13:08:56.707 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-grafana-dashboards-2:20.2.0-720.g2fc421c8.el1 76/86 2026-02-19T13:08:56.708 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mgr-cephadm-2:20.2.0-720.g2fc421c8.el10.noarc 77/86 2026-02-19T13:08:57.403 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.2.0-720.g2fc421c8.el10.noarc 77/86 2026-02-19T13:08:57.408 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mgr-dashboard-2:20.2.0-720.g2fc421c8.el10.noa 78/86 2026-02-19T13:08:57.760 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.2.0-720.g2fc421c8.el10.noa 78/86 2026-02-19T13:08:57.761 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mgr-diskprediction-local-2:20.2.0-720.g2fc421 79/86 2026-02-19T13:08:57.772 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.2.0-720.g2fc421 79/86 2026-02-19T13:08:57.772 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mgr-k8sevents-2:20.2.0-720.g2fc421c8.el10.noa 80/86 2026-02-19T13:08:57.805 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.2.0-720.g2fc421c8.el10.noa 80/86 2026-02-19T13:08:57.838 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mgr-modules-core-2:20.2.0-720.g2fc421c8.el10. 81/86 2026-02-19T13:08:57.839 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mgr-2:20.2.0-720.g2fc421c8.el10.x86_64 82/86 2026-02-19T13:08:57.861 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-mgr-2:20.2.0-720.g2fc421c8.el10.x86_64 82/86 2026-02-19T13:08:57.862 INFO:teuthology.orchestra.run.trial111.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-19T13:08:57.862 INFO:teuthology.orchestra.run.trial111.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-19T13:08:57.862 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-mgr.target' → '/usr/lib/systemd/system/ceph-mgr.target'. 2026-02-19T13:08:57.862 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-mgr.target' → '/usr/lib/systemd/system/ceph-mgr.target'. 2026-02-19T13:08:57.862 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:57.868 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mgr-rook-2:20.2.0-720.g2fc421c8.el10.noarch 83/86 2026-02-19T13:08:58.038 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-mgr-rook-2:20.2.0-720.g2fc421c8.el10.noarch 83/86 2026-02-19T13:08:58.040 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mon-2:20.2.0-720.g2fc421c8.el10.x86_64 84/86 2026-02-19T13:08:58.061 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-mon-2:20.2.0-720.g2fc421c8.el10.x86_64 84/86 2026-02-19T13:08:58.061 INFO:teuthology.orchestra.run.trial111.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-19T13:08:58.061 INFO:teuthology.orchestra.run.trial111.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-19T13:08:58.061 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-mon.target' → '/usr/lib/systemd/system/ceph-mon.target'. 2026-02-19T13:08:58.061 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-mon.target' → '/usr/lib/systemd/system/ceph-mon.target'. 2026-02-19T13:08:58.061 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:58.132 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-mds-2:20.2.0-720.g2fc421c8.el10.x86_64 85/86 2026-02-19T13:08:58.154 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-mds-2:20.2.0-720.g2fc421c8.el10.x86_64 85/86 2026-02-19T13:08:58.154 INFO:teuthology.orchestra.run.trial111.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-19T13:08:58.154 INFO:teuthology.orchestra.run.trial111.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-19T13:08:58.154 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-mds.target' → '/usr/lib/systemd/system/ceph-mds.target'. 2026-02-19T13:08:58.154 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-mds.target' → '/usr/lib/systemd/system/ceph-mds.target'. 2026-02-19T13:08:58.154 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:58.174 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-2:20.2.0-720.g2fc421c8.el10.x86_64 86/86 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-2:20.2.0-720.g2fc421c8.el10.x86_64 86/86 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-grafana-dashboards-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mds-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-cephadm-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-dashboard-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-diskprediction-local-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-k8sevents-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-modules-core-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mgr-rook-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-mon-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-osd-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-prometheus-alerts-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ceph-volume-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: cephadm-2:20.2.0-720.g2fc421c8.el10.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: cryptsetup-2.7.5-2.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: flexiblas-3.4.2-3.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: flexiblas-netlib-3.4.2-3.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: flexiblas-openblas-openmp-3.4.2-3.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: grpc-data-1.48.4-44.el10_0.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: ledmon-libs-1.1.0-2.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: libconfig-1.7.3-10.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: libgfortran-14.3.1-2.1.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: libquadmath-14.3.1-2.1.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: libstoragemgmt-1.10.1-4.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: libtool-ltdl-2.4.7-13.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: lua-5.4.6-7.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: lua-devel-5.4.6-7.el10.x86_64 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: luarocks-3.9.2-6.el10_0.noarch 2026-02-19T13:08:58.907 INFO:teuthology.orchestra.run.trial111.stdout: openblas-0.3.29-2.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: openblas-openmp-0.3.29-2.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: protobuf-compiler-3.19.6-11.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-asyncssh-2.17.0-1.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-autocommand-2.2.2-9.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-bcrypt-4.3.0-3.el10_1.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-cachetools-5.4.0-6.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-certifi-2023.05.07-6.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-cffi-1.16.0-7.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-cheroot-10.0.1-4.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-cherrypy-18.10.0-4.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-cryptography-43.0.0-4.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-devel-3.12.12-3.el10_1.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-durationpy-0.9-2.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-google-auth-1:2.40.3-1.el10_1.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-grpcio-1.48.4-44.el10_0.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-grpcio-tools-1.48.4-44.el10_0.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-isodate-0.6.1-14.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-jaraco-collections-5.1.0-1.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-jaraco-context-6.0.1-4.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-jaraco-functools-4.0.2-4.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-jaraco-text-4.0.0-3.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-jinja2-3.1.6-1.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-jmespath-1.0.1-8.el10.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-kubernetes-1:32.0.1-1.el10_1.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-libstoragemgmt-1.10.1-4.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-lxml-5.2.1-4.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-markupsafe-2.1.3-6.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-more-itertools-10.1.0-8.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-natsort-8.4.0-2.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-numpy-1:1.26.4-5.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-numpy-f2py-1:1.26.4-5.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-packaging-24.2-2.el10.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-ply-3.11-25.el10.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-portend-3.2.0-6.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-protobuf-3.19.6-11.el10.x86_64 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-pyOpenSSL-25.0.0-1.el10_1.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-pyasn1-0.6.1-1.el10.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-pyasn1-modules-0.6.1-1.el10.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-pycparser-2.20-16.el10.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-repoze-lru-0.7-23.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-requests-2.32.4-1.el10_0.noarch 2026-02-19T13:08:58.908 INFO:teuthology.orchestra.run.trial111.stdout: python3-requests-oauthlib-1.3.1-11.el10.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-routes-2.5.1-11.el10_0.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-rsa-4.9-7.el10_0.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-saml-1.16.0-11.el10.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-scipy-1.11.3-17.el10.x86_64 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-tempora-5.7.0-1.el10_0.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-urllib3-1.26.19-2.el10_1.1.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-websocket-client-1.8.0-2.el10_0.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-xmlsec-1.3.16-1.el10.x86_64 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-xmltodict-0.13.0-5.el10_0.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: python3-zc-lockfile-3.0.post1-11.el10_0.noarch 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: unzip-6.0-69.el10.x86_64 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: xmlsec1-1:1.2.39-3.el10.x86_64 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: xmlsec1-openssl-1:1.2.39-3.el10.x86_64 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: zip-3.0-45.el10.x86_64 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:58.909 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:08:58.958 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-base 2026-02-19T13:08:59.252 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:41 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:08:59.278 INFO:teuthology.orchestra.run.trial111.stdout:Package ceph-base-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:08:59.288 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:08:59.289 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:08:59.289 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:08:59.309 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install cephadm 2026-02-19T13:08:59.597 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:41 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:08:59.623 INFO:teuthology.orchestra.run.trial111.stdout:Package cephadm-2:20.2.0-720.g2fc421c8.el10.noarch is already installed. 2026-02-19T13:08:59.633 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:08:59.634 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:08:59.634 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:08:59.653 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-immutable-object-cache 2026-02-19T13:08:59.937 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:41 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:08:59.976 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:08:59.976 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:59.976 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repo Size 2026-02-19T13:08:59.976 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout: ceph-immutable-object-cache x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 156 k 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout:Install 1 Package 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 156 k 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 441 k 2026-02-19T13:08:59.977 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:09:00.429 INFO:teuthology.orchestra.run.trial111.stdout:ceph-immutable-object-cache-20.2.0-720.g2fc421c 344 kB/s | 156 kB 00:00 2026-02-19T13:09:00.430 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:09:00.430 INFO:teuthology.orchestra.run.trial111.stdout:Total 343 kB/s | 156 kB 00:00 2026-02-19T13:09:00.430 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:09:00.434 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:09:00.435 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:09:00.464 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:09:00.464 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:09:00.515 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:09:00.517 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-immutable-object-cache-2:20.2.0-720.g2fc421c8.e 1/1 2026-02-19T13:09:00.540 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.2.0-720.g2fc421c8.e 1/1 2026-02-19T13:09:00.540 INFO:teuthology.orchestra.run.trial111.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-19T13:09:00.540 INFO:teuthology.orchestra.run.trial111.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-19T13:09:00.540 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:00.821 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:00.822 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:09:00.822 INFO:teuthology.orchestra.run.trial111.stdout: ceph-immutable-object-cache-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:09:00.822 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:00.822 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:00.848 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-mgr 2026-02-19T13:09:01.139 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:43 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:01.165 INFO:teuthology.orchestra.run.trial111.stdout:Package ceph-mgr-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:09:01.175 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:01.175 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:01.176 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:01.196 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-mgr-dashboard 2026-02-19T13:09:01.488 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:43 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:01.515 INFO:teuthology.orchestra.run.trial111.stdout:Package ceph-mgr-dashboard-2:20.2.0-720.g2fc421c8.el10.noarch is already installed. 2026-02-19T13:09:01.525 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:01.525 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:01.525 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:01.545 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-19T13:09:01.831 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:43 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:01.857 INFO:teuthology.orchestra.run.trial111.stdout:Package ceph-mgr-diskprediction-local-2:20.2.0-720.g2fc421c8.el10.noarch is already installed. 2026-02-19T13:09:01.867 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:01.868 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:01.868 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:01.887 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-mgr-rook 2026-02-19T13:09:02.169 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:44 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:02.197 INFO:teuthology.orchestra.run.trial111.stdout:Package ceph-mgr-rook-2:20.2.0-720.g2fc421c8.el10.noarch is already installed. 2026-02-19T13:09:02.208 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:02.208 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:02.209 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:02.228 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-mgr-cephadm 2026-02-19T13:09:02.518 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:44 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:02.545 INFO:teuthology.orchestra.run.trial111.stdout:Package ceph-mgr-cephadm-2:20.2.0-720.g2fc421c8.el10.noarch is already installed. 2026-02-19T13:09:02.555 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:02.555 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:02.556 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:02.576 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-fuse 2026-02-19T13:09:02.862 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:44 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:02.900 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repo Size 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout: ceph-fuse x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 1.0 M 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:02.901 INFO:teuthology.orchestra.run.trial111.stdout:Install 1 Package 2026-02-19T13:09:02.902 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:02.902 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 1.0 M 2026-02-19T13:09:02.902 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 2.9 M 2026-02-19T13:09:02.902 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:09:03.647 INFO:teuthology.orchestra.run.trial111.stdout:ceph-fuse-20.2.0-720.g2fc421c8.el10.x86_64.rpm 1.3 MB/s | 1.0 MB 00:00 2026-02-19T13:09:03.648 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:09:03.648 INFO:teuthology.orchestra.run.trial111.stdout:Total 1.3 MB/s | 1.0 MB 00:00 2026-02-19T13:09:03.648 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:09:03.653 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:09:03.654 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:09:03.686 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:09:03.686 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:09:03.765 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:09:03.785 INFO:teuthology.orchestra.run.trial111.stdout: Installing : ceph-fuse-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:04.068 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: ceph-fuse-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:04.068 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:04.069 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:09:04.069 INFO:teuthology.orchestra.run.trial111.stdout: ceph-fuse-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:09:04.069 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:04.069 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:04.092 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install ceph-volume 2026-02-19T13:09:04.388 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:46 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:04.417 INFO:teuthology.orchestra.run.trial111.stdout:Package ceph-volume-2:20.2.0-720.g2fc421c8.el10.noarch is already installed. 2026-02-19T13:09:04.426 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:04.427 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:04.427 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:04.447 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install librados-devel 2026-02-19T13:09:04.738 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:46 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:04.777 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:04.777 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repo Size 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout: librados-devel x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 128 k 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout:Install 1 Package 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:04.778 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 128 k 2026-02-19T13:09:04.779 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 448 k 2026-02-19T13:09:04.779 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:09:05.170 INFO:teuthology.orchestra.run.trial111.stdout:librados-devel-20.2.0-720.g2fc421c8.el10.x86_64 325 kB/s | 128 kB 00:00 2026-02-19T13:09:05.171 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:09:05.171 INFO:teuthology.orchestra.run.trial111.stdout:Total 324 kB/s | 128 kB 00:00 2026-02-19T13:09:05.171 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:09:05.175 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:09:05.175 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:09:05.204 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:09:05.205 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:09:05.256 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:09:05.274 INFO:teuthology.orchestra.run.trial111.stdout: Installing : librados-devel-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:05.386 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: librados-devel-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:05.386 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:05.386 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:09:05.386 INFO:teuthology.orchestra.run.trial111.stdout: librados-devel-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:09:05.386 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:05.386 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:05.413 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install libcephfs2 2026-02-19T13:09:05.703 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:47 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:05.731 INFO:teuthology.orchestra.run.trial111.stdout:Package libcephfs2-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:09:05.740 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:05.741 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:05.741 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:05.761 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install libcephfs-devel 2026-02-19T13:09:06.045 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:48 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:06.083 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:06.084 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:06.084 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repo Size 2026-02-19T13:09:06.084 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:06.084 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:09:06.084 INFO:teuthology.orchestra.run.trial111.stdout: libcephfs-devel x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 34 k 2026-02-19T13:09:06.085 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:06.085 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:09:06.085 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:06.085 INFO:teuthology.orchestra.run.trial111.stdout:Install 1 Package 2026-02-19T13:09:06.085 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:06.085 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 34 k 2026-02-19T13:09:06.085 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 155 k 2026-02-19T13:09:06.085 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:09:06.347 INFO:teuthology.orchestra.run.trial111.stdout:libcephfs-devel-20.2.0-720.g2fc421c8.el10.x86_6 131 kB/s | 34 kB 00:00 2026-02-19T13:09:06.347 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:09:06.347 INFO:teuthology.orchestra.run.trial111.stdout:Total 131 kB/s | 34 kB 00:00 2026-02-19T13:09:06.347 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:09:06.349 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:09:06.350 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:09:06.352 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:09:06.352 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:09:06.372 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:09:06.391 INFO:teuthology.orchestra.run.trial111.stdout: Installing : libcephfs-devel-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:06.489 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: libcephfs-devel-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:06.489 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:06.489 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:09:06.489 INFO:teuthology.orchestra.run.trial111.stdout: libcephfs-devel-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:09:06.489 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:06.489 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:06.512 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install librados2 2026-02-19T13:09:06.801 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:48 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:06.828 INFO:teuthology.orchestra.run.trial111.stdout:Package librados2-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:09:06.838 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:06.838 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:06.839 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:06.859 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install librbd1 2026-02-19T13:09:07.145 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:49 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:07.173 INFO:teuthology.orchestra.run.trial111.stdout:Package librbd1-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:09:07.183 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:07.183 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:07.184 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:07.203 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install python3-rados 2026-02-19T13:09:07.491 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:49 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:07.518 INFO:teuthology.orchestra.run.trial111.stdout:Package python3-rados-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:09:07.527 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:07.528 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:07.528 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:07.549 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install python3-rgw 2026-02-19T13:09:07.836 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:49 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:07.861 INFO:teuthology.orchestra.run.trial111.stdout:Package python3-rgw-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:09:07.871 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:07.871 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:07.871 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:07.891 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install python3-cephfs 2026-02-19T13:09:08.172 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:50 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:08.201 INFO:teuthology.orchestra.run.trial111.stdout:Package python3-cephfs-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:09:08.210 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:08.211 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:08.211 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:08.230 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install python3-rbd 2026-02-19T13:09:08.519 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:50 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:08.545 INFO:teuthology.orchestra.run.trial111.stdout:Package python3-rbd-2:20.2.0-720.g2fc421c8.el10.x86_64 is already installed. 2026-02-19T13:09:08.554 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:08.555 INFO:teuthology.orchestra.run.trial111.stdout:Nothing to do. 2026-02-19T13:09:08.555 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:08.576 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install rbd-fuse 2026-02-19T13:09:08.863 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:50 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:08.900 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout: Package Architecture Version Repository Size 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout: rbd-fuse x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 93 k 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:08.901 INFO:teuthology.orchestra.run.trial111.stdout:Install 1 Package 2026-02-19T13:09:08.902 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:08.902 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 93 k 2026-02-19T13:09:08.902 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 228 k 2026-02-19T13:09:08.902 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:09:09.186 INFO:teuthology.orchestra.run.trial111.stdout:rbd-fuse-20.2.0-720.g2fc421c8.el10.x86_64.rpm 325 kB/s | 93 kB 00:00 2026-02-19T13:09:09.186 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:09:09.186 INFO:teuthology.orchestra.run.trial111.stdout:Total 324 kB/s | 93 kB 00:00 2026-02-19T13:09:09.186 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:09:09.191 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:09:09.191 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:09:09.221 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:09:09.222 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:09:09.270 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:09:09.289 INFO:teuthology.orchestra.run.trial111.stdout: Installing : rbd-fuse-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:09.403 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: rbd-fuse-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:09.403 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:09.403 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:09:09.403 INFO:teuthology.orchestra.run.trial111.stdout: rbd-fuse-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:09:09.403 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:09.403 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:09.430 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install rbd-mirror 2026-02-19T13:09:09.716 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:51 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:09.753 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout: Package Arch Version Repo Size 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout: rbd-mirror x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 2.9 M 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:09.754 INFO:teuthology.orchestra.run.trial111.stdout:Install 1 Package 2026-02-19T13:09:09.755 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:09.755 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 2.9 M 2026-02-19T13:09:09.755 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 11 M 2026-02-19T13:09:09.755 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:09:10.793 INFO:teuthology.orchestra.run.trial111.stdout:rbd-mirror-20.2.0-720.g2fc421c8.el10.x86_64.rpm 2.8 MB/s | 2.9 MB 00:01 2026-02-19T13:09:10.794 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:09:10.794 INFO:teuthology.orchestra.run.trial111.stdout:Total 2.8 MB/s | 2.9 MB 00:01 2026-02-19T13:09:10.794 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:09:10.799 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:09:10.799 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:09:10.837 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:09:10.837 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:09:10.983 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:09:10.985 INFO:teuthology.orchestra.run.trial111.stdout: Installing : rbd-mirror-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:11.007 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: rbd-mirror-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:11.008 INFO:teuthology.orchestra.run.trial111.stdout:Glob pattern passed to preset, but globs are not supported for this. 2026-02-19T13:09:11.008 INFO:teuthology.orchestra.run.trial111.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-19T13:09:11.008 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target' → '/usr/lib/systemd/system/ceph-rbd-mirror.target'. 2026-02-19T13:09:11.008 INFO:teuthology.orchestra.run.trial111.stdout:Created symlink '/etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target' → '/usr/lib/systemd/system/ceph-rbd-mirror.target'. 2026-02-19T13:09:11.008 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:11.312 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:11.312 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:09:11.313 INFO:teuthology.orchestra.run.trial111.stdout: rbd-mirror-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:09:11.313 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:11.313 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:11.340 DEBUG:teuthology.orchestra.run.trial111:> sudo yum -y install rbd-nbd 2026-02-19T13:09:11.633 INFO:teuthology.orchestra.run.trial111.stdout:Last metadata expiration check: 0:00:53 ago on Thu 19 Feb 2026 01:08:18 PM UTC. 2026-02-19T13:09:11.670 INFO:teuthology.orchestra.run.trial111.stdout:Dependencies resolved. 2026-02-19T13:09:11.670 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:11.670 INFO:teuthology.orchestra.run.trial111.stdout: Package Architecture Version Repository Size 2026-02-19T13:09:11.670 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:11.670 INFO:teuthology.orchestra.run.trial111.stdout:Installing: 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout: rbd-nbd x86_64 2:20.2.0-720.g2fc421c8.el10 ceph 184 k 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout:Transaction Summary 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout:================================================================================ 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout:Install 1 Package 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout:Total download size: 184 k 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout:Installed size: 492 k 2026-02-19T13:09:11.671 INFO:teuthology.orchestra.run.trial111.stdout:Downloading Packages: 2026-02-19T13:09:12.088 INFO:teuthology.orchestra.run.trial111.stdout:rbd-nbd-20.2.0-720.g2fc421c8.el10.x86_64.rpm 440 kB/s | 184 kB 00:00 2026-02-19T13:09:12.089 INFO:teuthology.orchestra.run.trial111.stdout:-------------------------------------------------------------------------------- 2026-02-19T13:09:12.089 INFO:teuthology.orchestra.run.trial111.stdout:Total 438 kB/s | 184 kB 00:00 2026-02-19T13:09:12.089 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction check 2026-02-19T13:09:12.094 INFO:teuthology.orchestra.run.trial111.stdout:Transaction check succeeded. 2026-02-19T13:09:12.094 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction test 2026-02-19T13:09:12.123 INFO:teuthology.orchestra.run.trial111.stdout:Transaction test succeeded. 2026-02-19T13:09:12.123 INFO:teuthology.orchestra.run.trial111.stdout:Running transaction 2026-02-19T13:09:12.178 INFO:teuthology.orchestra.run.trial111.stdout: Preparing : 1/1 2026-02-19T13:09:12.198 INFO:teuthology.orchestra.run.trial111.stdout: Installing : rbd-nbd-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:12.316 INFO:teuthology.orchestra.run.trial111.stdout: Running scriptlet: rbd-nbd-2:20.2.0-720.g2fc421c8.el10.x86_64 1/1 2026-02-19T13:09:12.316 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:12.316 INFO:teuthology.orchestra.run.trial111.stdout:Installed: 2026-02-19T13:09:12.316 INFO:teuthology.orchestra.run.trial111.stdout: rbd-nbd-2:20.2.0-720.g2fc421c8.el10.x86_64 2026-02-19T13:09:12.316 INFO:teuthology.orchestra.run.trial111.stdout: 2026-02-19T13:09:12.316 INFO:teuthology.orchestra.run.trial111.stdout:Complete! 2026-02-19T13:09:12.347 DEBUG:teuthology.parallel:result is None 2026-02-19T13:09:12.348 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=rocky%2F10%2Fx86_64&sha1=2fc421c83f0b86ea21b9e4289cad0f7957e09eb1 2026-02-19T13:09:12.516 DEBUG:teuthology.orchestra.run.trial111:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-19T13:09:12.537 INFO:teuthology.orchestra.run.trial111.stdout:20.2.0-720.g2fc421c8.el10 2026-02-19T13:09:12.538 INFO:teuthology.packaging:The installed version of ceph is 20.2.0-720.g2fc421c8.el10 2026-02-19T13:09:12.538 INFO:teuthology.task.install:The correct ceph version 20.2.0-720.g2fc421c8 is installed. 2026-02-19T13:09:12.541 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-19T13:09:12.541 DEBUG:teuthology.orchestra.run.trial111:> set -ex 2026-02-19T13:09:12.541 DEBUG:teuthology.orchestra.run.trial111:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-19T13:09:12.605 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-19T13:09:12.605 DEBUG:teuthology.orchestra.run.trial111:> set -ex 2026-02-19T13:09:12.606 DEBUG:teuthology.orchestra.run.trial111:> sudo dd of=/usr/bin/daemon-helper 2026-02-19T13:09:12.672 DEBUG:teuthology.orchestra.run.trial111:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-19T13:09:12.738 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-19T13:09:12.739 DEBUG:teuthology.orchestra.run.trial111:> set -ex 2026-02-19T13:09:12.739 DEBUG:teuthology.orchestra.run.trial111:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-19T13:09:12.805 DEBUG:teuthology.orchestra.run.trial111:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-19T13:09:12.872 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-19T13:09:12.873 DEBUG:teuthology.orchestra.run.trial111:> set -ex 2026-02-19T13:09:12.873 DEBUG:teuthology.orchestra.run.trial111:> sudo dd of=/usr/bin/stdin-killer 2026-02-19T13:09:12.940 DEBUG:teuthology.orchestra.run.trial111:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-19T13:09:13.006 INFO:teuthology.run_tasks:Running task workunit... 2026-02-19T13:09:13.030 INFO:tasks.workunit:Pulling workunits from ref 624b9a7bcb2064ce17206482df6a1da007ff7675 2026-02-19T13:09:13.031 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-19T13:09:13.031 INFO:tasks.workunit:timeout=3h 2026-02-19T13:09:13.031 INFO:tasks.workunit:cleanup=True 2026-02-19T13:09:13.032 DEBUG:teuthology.orchestra.run.trial111:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-19T13:09:13.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T13:09:13.064 INFO:teuthology.orchestra.run.trial111.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-19T13:09:13.065 DEBUG:teuthology.orchestra.run.trial111:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-19T13:09:13.122 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-19T13:09:13.123 DEBUG:teuthology.orchestra.run.trial111:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-19T13:09:13.182 DEBUG:teuthology.orchestra.run.trial111:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/NitzanMordhai/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 624b9a7bcb2064ce17206482df6a1da007ff7675 2026-02-19T13:09:13.245 INFO:tasks.workunit.client.0.trial111.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-19T13:09:32.644 INFO:tasks.workunit.client.0.trial111.stderr:Note: switching to '624b9a7bcb2064ce17206482df6a1da007ff7675'. 2026-02-19T13:09:32.644 INFO:tasks.workunit.client.0.trial111.stderr: 2026-02-19T13:09:32.644 INFO:tasks.workunit.client.0.trial111.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-19T13:09:32.644 INFO:tasks.workunit.client.0.trial111.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr:state without impacting any branches by switching back to a branch. 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr: 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr: 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr: git switch -c 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr: 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr:Or undo this operation with: 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr: 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr: git switch - 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr: 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr: 2026-02-19T13:09:32.645 INFO:tasks.workunit.client.0.trial111.stderr:HEAD is now at 624b9a7bcb2 Integration branch metadata 2026-02-19T13:09:32.650 DEBUG:teuthology.orchestra.run.trial111:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/standalone && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-19T13:09:32.709 DEBUG:teuthology.orchestra.run.trial111:> set -ex 2026-02-19T13:09:32.709 DEBUG:teuthology.orchestra.run.trial111:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-19T13:09:32.767 INFO:tasks.workunit:Running workunits matching erasure-code on client.0... 2026-02-19T13:09:32.768 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code-plugins.sh... 2026-02-19T13:09:32.769 DEBUG:teuthology.orchestra.run.trial111:workunit test erasure-code/test-erasure-code-plugins.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh 2026-02-19T13:09:32.830 INFO:tasks.workunit.client.0.trial111.stderr:+ source /home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh 2026-02-19T13:09:32.831 INFO:tasks.workunit.client.0.trial111.stderr:++ TIMEOUT=300 2026-02-19T13:09:32.831 INFO:tasks.workunit.client.0.trial111.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-19T13:09:32.831 INFO:tasks.workunit.client.0.trial111.stderr:++ MAX_TIMEOUT=15 2026-02-19T13:09:32.831 INFO:tasks.workunit.client.0.trial111.stderr:++ PG_NUM=4 2026-02-19T13:09:32.831 INFO:tasks.workunit.client.0.trial111.stderr:++ TMPDIR=/tmp 2026-02-19T13:09:32.831 INFO:tasks.workunit.client.0.trial111.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp 2026-02-19T13:09:32.831 INFO:tasks.workunit.client.0.trial111.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-19T13:09:32.831 INFO:tasks.workunit.client.0.trial111.stderr:+++ uname 2026-02-19T13:09:32.833 INFO:tasks.workunit.client.0.trial111.stderr:++ '[' Linux = FreeBSD ']' 2026-02-19T13:09:32.833 INFO:tasks.workunit.client.0.trial111.stderr:++ SED=sed 2026-02-19T13:09:32.833 INFO:tasks.workunit.client.0.trial111.stderr:++ AWK=awk 2026-02-19T13:09:32.833 INFO:tasks.workunit.client.0.trial111.stderr:+++ stty -a 2026-02-19T13:09:32.833 INFO:tasks.workunit.client.0.trial111.stderr:+++ head -1 2026-02-19T13:09:32.833 INFO:tasks.workunit.client.0.trial111.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-19T13:09:32.833 INFO:tasks.workunit.client.0.trial111.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T13:09:32.834 INFO:tasks.workunit.client.0.trial111.stderr:++ termwidth= 2026-02-19T13:09:32.834 INFO:tasks.workunit.client.0.trial111.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-19T13:09:32.834 INFO:tasks.workunit.client.0.trial111.stderr:++ DIFFCOLOPTS='-y ' 2026-02-19T13:09:32.834 INFO:tasks.workunit.client.0.trial111.stderr:++ KERNCORE=kernel.core_pattern 2026-02-19T13:09:32.834 INFO:tasks.workunit.client.0.trial111.stderr:++ EXTRA_OPTS= 2026-02-19T13:09:32.835 INFO:tasks.workunit.client.0.trial111.stderr:++ test '' = TESTS 2026-02-19T13:09:32.835 INFO:tasks.workunit.client.0.trial111.stderr:++ uname -m 2026-02-19T13:09:32.837 INFO:tasks.workunit.client.0.trial111.stderr:+ arch=x86_64 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ case $arch in 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ legacy_jerasure_plugins=(jerasure_generic jerasure_sse3 jerasure_sse4) 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ legacy_shec_plugins=(shec_generic shec_sse3 shec_sse4) 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ plugins=(jerasure shec lrc isa) 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ main test-erasure-code-plugins 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ local dir=td/test-erasure-code-plugins 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ shift 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ shopt -s -o xtrace 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T13:09:32.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/test-erasure-code-plugins 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:26: run: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:27: run: shift 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: export CEPH_MON=127.0.0.1:17110 2026-02-19T13:09:32.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:29: run: CEPH_MON=127.0.0.1:17110 2026-02-19T13:09:32.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:30: run: export CEPH_ARGS 2026-02-19T13:09:32.840 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: uuidgen 2026-02-19T13:09:32.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:31: run: CEPH_ARGS+='--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none ' 2026-02-19T13:09:32.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:32: run: CEPH_ARGS+='--mon-host=127.0.0.1:17110 ' 2026-02-19T13:09:32.842 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: set 2026-02-19T13:09:32.842 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T13:09:32.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:34: run: local 'funcs=TEST_ec_profile_warning 2026-02-19T13:09:32.842 INFO:tasks.workunit.client.0.trial111.stderr:TEST_preload_no_warning 2026-02-19T13:09:32.842 INFO:tasks.workunit.client.0.trial111.stderr:TEST_preload_no_warning_default 2026-02-19T13:09:32.842 INFO:tasks.workunit.client.0.trial111.stderr:TEST_preload_warning' 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_ec_profile_warning td/test-erasure-code-plugins 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:92: TEST_ec_profile_warning: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:94: TEST_ec_profile_warning: setup td/test-erasure-code-plugins 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:32.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:32.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:32.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:32.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:32.844 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:32.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:32.845 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:32.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:32.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:32.845 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:32.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:32.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:32.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:32.846 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:32.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:32.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:32.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:32.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:32.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:32.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:32.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:32.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:32.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:32.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:09:32.850 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:09:32.850 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:32.850 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:32.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:09:32.851 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:09:32.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T13:09:32.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:95: TEST_ec_profile_warning: run_mon td/test-erasure-code-plugins a 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:09:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-19T13:09:32.880 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:09:32.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:32.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:32.881 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:32.881 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:32.881 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:32.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:32.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:09:32.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:32.907 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:32.908 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:32.908 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:32.908 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:32.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:32.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:32.968 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:32.969 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:32.969 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:32.969 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:32.969 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:32.969 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:09:33.027 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:96: TEST_ec_profile_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:09:33.027 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:33.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:09:33.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:09:33.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:09:33.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:09:33.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:09:33.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:09:33.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:33.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:33.141 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:33.141 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:33.141 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:33.141 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:33.141 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:33.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:33.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: seq 0 2 2026-02-19T13:09:33.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-19T13:09:33.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 0 2026-02-19T13:09:33.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:33.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:33.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:09:33.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:33.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:33.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:09:33.164 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:33.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3606dc6a-0cf2-4ed7-b881-a3c45dea8d8c 2026-02-19T13:09:33.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3606dc6a-0cf2-4ed7-b881-a3c45dea8d8c' 2026-02-19T13:09:33.165 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 3606dc6a-0cf2-4ed7-b881-a3c45dea8d8c 2026-02-19T13:09:33.166 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:33.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCNC5dpq9lbChAAMGG1h3a81+V2DqCHdMs18A== 2026-02-19T13:09:33.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCNC5dpq9lbChAAMGG1h3a81+V2DqCHdMs18A=="}' 2026-02-19T13:09:33.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3606dc6a-0cf2-4ed7-b881-a3c45dea8d8c -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:33.281 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:33.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:33.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCNC5dpq9lbChAAMGG1h3a81+V2DqCHdMs18A== --osd-uuid 3606dc6a-0cf2-4ed7-b881-a3c45dea8d8c 2026-02-19T13:09:33.306 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:33.309+0000 7f33ebc6a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:33.308 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:33.310+0000 7f33ebc6a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:33.308 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:33.311+0000 7f33ebc6a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:33.309 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:33.312+0000 7f33ebc6a900 -1 bdev(0x55bc25fe4c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:33.309 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:33.312+0000 7f33ebc6a900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:09:33.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:09:33.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:33.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:09:33.808 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:09:33.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:33.920 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:09:33.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:09:33.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:33.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:33.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:33.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:33.934 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:33.936+0000 7f9d5e035900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:33.935 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:33.938+0000 7f9d5e035900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:33.936 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:33.939+0000 7f9d5e035900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:34.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:09:34.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:34.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:09:34.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:34.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:34.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:34.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:34.029 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:34.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:34.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:34.143 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:34.145+0000 7f9d5e035900 -1 Falling back to public interface 2026-02-19T13:09:34.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:34.501 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:34.503+0000 7f9d5e035900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:09:35.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:35.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:35.248 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:09:35.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:35.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:35.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:35.465 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3672673615,v1:127.0.0.1:6803/3672673615] [v2:127.0.0.1:6804/3672673615,v1:127.0.0.1:6805/3672673615] exists,up 3606dc6a-0cf2-4ed7-b881-a3c45dea8d8c 2026-02-19T13:09:35.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:35.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:35.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 1 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/1 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:35.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/1' 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/1/journal' 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:35.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:09:35.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/1 2026-02-19T13:09:35.473 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:35.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=beec7a3d-5360-4858-bfcf-cd4a0aebb64d 2026-02-19T13:09:35.473 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 beec7a3d-5360-4858-bfcf-cd4a0aebb64d 2026-02-19T13:09:35.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 beec7a3d-5360-4858-bfcf-cd4a0aebb64d' 2026-02-19T13:09:35.474 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:35.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCPC5dpIQiyHBAA7JITrs74cpG/2idAUAcqaA== 2026-02-19T13:09:35.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCPC5dpIQiyHBAA7JITrs74cpG/2idAUAcqaA=="}' 2026-02-19T13:09:35.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new beec7a3d-5360-4858-bfcf-cd4a0aebb64d -i td/test-erasure-code-plugins/1/new.json 2026-02-19T13:09:35.698 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:09:35.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/1/new.json 2026-02-19T13:09:35.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCPC5dpIQiyHBAA7JITrs74cpG/2idAUAcqaA== --osd-uuid beec7a3d-5360-4858-bfcf-cd4a0aebb64d 2026-02-19T13:09:35.724 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:35.727+0000 7f3c91055900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:35.725 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:35.728+0000 7f3c91055900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:35.726 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:35.729+0000 7f3c91055900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:35.727 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:35.729+0000 7f3c91055900 -1 bdev(0x55f5fc7aec00 td/test-erasure-code-plugins/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:35.727 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:35.730+0000 7f3c91055900 -1 bluestore(td/test-erasure-code-plugins/1) _read_fsid unparsable uuid 2026-02-19T13:09:36.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/1/keyring 2026-02-19T13:09:36.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:36.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:09:36.119 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:09:36.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:36.419 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:09:36.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:09:36.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/1 --osd-journal=td/test-erasure-code-plugins/1/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:36.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:36.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:36.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:36.433 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:36.436+0000 7f4563630900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:36.434 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:36.437+0000 7f4563630900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:36.435 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:36.438+0000 7f4563630900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:36.631 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:36.633+0000 7f4563630900 -1 Falling back to public interface 2026-02-19T13:09:36.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:09:36.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:36.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:09:36.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:36.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:36.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:36.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:36.641 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:36.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:36.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:36.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:36.910 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:36.913+0000 7f4563630900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:09:37.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:37.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:37.863 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:09:37.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:37.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:37.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:09:38.082 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1818664585,v1:127.0.0.1:6811/1818664585] [v2:127.0.0.1:6812/1818664585,v1:127.0.0.1:6813/1818664585] exists,up beec7a3d-5360-4858-bfcf-cd4a0aebb64d 2026-02-19T13:09:38.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:97: TEST_ec_profile_warning: for id in $(seq 0 2) 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:98: TEST_ec_profile_warning: run_osd td/test-erasure-code-plugins 2 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/2 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/2' 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/2/journal' 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:38.088 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:09:38.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/2 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=28a3f922-c8d9-453b-81e7-a006878b3de1 2026-02-19T13:09:38.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 28a3f922-c8d9-453b-81e7-a006878b3de1' 2026-02-19T13:09:38.091 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 28a3f922-c8d9-453b-81e7-a006878b3de1 2026-02-19T13:09:38.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:38.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCSC5dp26jnBRAAsQk+SZME1iTyJHpyutCsRw== 2026-02-19T13:09:38.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCSC5dp26jnBRAAsQk+SZME1iTyJHpyutCsRw=="}' 2026-02-19T13:09:38.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 28a3f922-c8d9-453b-81e7-a006878b3de1 -i td/test-erasure-code-plugins/2/new.json 2026-02-19T13:09:38.316 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:09:38.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/2/new.json 2026-02-19T13:09:38.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCSC5dp26jnBRAAsQk+SZME1iTyJHpyutCsRw== --osd-uuid 28a3f922-c8d9-453b-81e7-a006878b3de1 2026-02-19T13:09:38.342 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:38.344+0000 7fd694937900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:38.343 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:38.346+0000 7fd694937900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:38.344 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:38.346+0000 7fd694937900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:38.344 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:38.347+0000 7fd694937900 -1 bdev(0x55efd06b8c00 td/test-erasure-code-plugins/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:38.344 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:38.347+0000 7fd694937900 -1 bluestore(td/test-erasure-code-plugins/2) _read_fsid unparsable uuid 2026-02-19T13:09:38.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/2/keyring 2026-02-19T13:09:38.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:38.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:09:38.760 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:09:38.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:39.056 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:09:39.056 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:09:39.056 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/2 --osd-journal=td/test-erasure-code-plugins/2/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:09:39.056 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:39.056 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:39.057 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:39.070 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:39.073+0000 7f21ce00c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:39.071 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:39.074+0000 7f21ce00c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:39.072 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:39.075+0000 7f21ce00c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:39.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:09:39.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:39.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:09:39.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:39.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:39.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:39.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:39.274 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:39.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:39.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:39.303 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:39.304+0000 7f21ce00c900 -1 Falling back to public interface 2026-02-19T13:09:39.492 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:39.624 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:39.625+0000 7f21ce00c900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:09:40.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:40.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:40.494 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:09:40.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:40.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:40.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:09:40.715 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4042752570,v1:127.0.0.1:6819/4042752570] [v2:127.0.0.1:6820/4042752570,v1:127.0.0.1:6821/4042752570] exists,up 28a3f922-c8d9-453b-81e7-a006878b3de1 2026-02-19T13:09:40.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:40.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:40.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:40.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:100: TEST_ec_profile_warning: create_rbd_pool 2026-02-19T13:09:40.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:09:40.926 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' does not exist 2026-02-19T13:09:40.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:09:40.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:09:41.163 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:09:41.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:09:42.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:09:42.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:101: TEST_ec_profile_warning: wait_for_clean 2026-02-19T13:09:42.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:09:42.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:09:42.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:09:42.456 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:09:42.456 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:09:42.456 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:09:42.456 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:09:42.456 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:09:42.456 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:09:42.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:09:42.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:09:42.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:09:42.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:09:42.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:09:42.529 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:09:42.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:09:42.747 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:09:42.747 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:09:42.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:09:42.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:42.748 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:09:42.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836484 2026-02-19T13:09:42.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836484 2026-02-19T13:09:42.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484' 2026-02-19T13:09:42.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:42.826 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:09:42.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705667 2026-02-19T13:09:42.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705667 2026-02-19T13:09:42.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667' 2026-02-19T13:09:42.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:09:42.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:09:42.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574851 2026-02-19T13:09:42.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574851 2026-02-19T13:09:42.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836484 1-38654705667 2-55834574851' 2026-02-19T13:09:42.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:42.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836484 2026-02-19T13:09:42.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:42.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:09:42.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836484 2026-02-19T13:09:42.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:42.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836484 2026-02-19T13:09:42.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836484' 2026-02-19T13:09:42.984 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 21474836484 2026-02-19T13:09:42.984 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:43.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836482 -lt 21474836484 2026-02-19T13:09:43.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:09:44.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:09:44.202 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:09:44.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836484 -lt 21474836484 2026-02-19T13:09:44.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:44.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705667 2026-02-19T13:09:44.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:44.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:09:44.427 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705667 2026-02-19T13:09:44.427 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:44.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705667 2026-02-19T13:09:44.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705667' 2026-02-19T13:09:44.429 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 38654705667 2026-02-19T13:09:44.429 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:09:44.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705668 -lt 38654705667 2026-02-19T13:09:44.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:09:44.651 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574851 2026-02-19T13:09:44.652 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:09:44.652 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:09:44.653 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574851 2026-02-19T13:09:44.653 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:09:44.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574851 2026-02-19T13:09:44.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574851' 2026-02-19T13:09:44.654 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 55834574851 2026-02-19T13:09:44.655 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:09:44.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574851 -lt 55834574851 2026-02-19T13:09:44.876 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:09:44.876 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:44.876 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:45.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:09:45.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:09:45.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:09:45.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:09:45.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:09:45.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:09:45.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:09:45.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:09:45.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:09:45.387 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:09:45.387 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:09:45.387 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:09:45.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:09:45.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:09:45.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:09:45.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-19T13:09:45.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_generic crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_generic 2026-02-19T13:09:45.963 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T13:09:45.963 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:45.963 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:45.964 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:45.964 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:45.964 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:45.964 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:45.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T13:09:45.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:46.016 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:46.023 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:46.026 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:45.842+0000 7f664a2f76c0 0 mon.a@0(leader).osd e19 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.026 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:45.952+0000 7f664a2f76c0 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.026 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:45.952+0000 7f664a2f76c0 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_generic uses plugin jerasure_generic that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.027 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-19T13:09:46.027 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse3 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse3 2026-02-19T13:09:46.311 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T13:09:46.311 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:46.311 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:46.311 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:46.311 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:46.311 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:46.312 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:46.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T13:09:46.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:46.366 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:46.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:46.377 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.190+0000 7f664a2f76c0 0 mon.a@0(leader).osd e20 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.377 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.300+0000 7f664a2f76c0 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.377 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.300+0000 7f664a2f76c0 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_sse3 uses plugin jerasure_sse3 that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:103: TEST_ec_profile_warning: for plugin in ${legacy_jerasure_plugins[*]} 2026-02-19T13:09:46.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:104: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-jerasure_sse4 crush-failure-domain=osd technique=reed_sol_van plugin=jerasure_sse4 2026-02-19T13:09:46.665 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T13:09:46.665 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:46.665 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:46.665 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:46.665 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:46.666 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:46.666 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:46.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T13:09:46.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:105: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:46.718 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:46.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:106: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:46.727 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.542+0000 7f664a2f76c0 0 mon.a@0(leader).osd e21 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.727 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.653+0000 7f664a2f76c0 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.727 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.653+0000 7f664a2f76c0 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-jerasure_sse4 uses plugin jerasure_sse4 that has been deprecated. Please use jerasure instead. 2026-02-19T13:09:46.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-19T13:09:46.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_generic crush-failure-domain=osd plugin=shec_generic 2026-02-19T13:09:47.008 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T13:09:47.009 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:47.009 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:47.009 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:47.009 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.009 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.009 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:47.009 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T13:09:47.009 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:47.062 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:47.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:47.071 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.888+0000 7f664a2f76c0 0 mon.a@0(leader).osd e22 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.072 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.997+0000 7f664a2f76c0 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.072 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:46.997+0000 7f664a2f76c0 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-shec_generic uses plugin shec_generic that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-19T13:09:47.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse3 crush-failure-domain=osd plugin=shec_sse3 2026-02-19T13:09:47.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T13:09:47.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:47.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:47.350 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:47.350 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.350 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:47.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T13:09:47.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:47.403 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:47.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:47.413 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:47.230+0000 7f664a2f76c0 0 mon.a@0(leader).osd e23 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.413 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:47.339+0000 7f664a2f76c0 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.413 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:47.339+0000 7f664a2f76c0 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_sse3 uses plugin shec_sse3 that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:109: TEST_ec_profile_warning: for plugin in ${legacy_shec_plugins[*]} 2026-02-19T13:09:47.414 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:110: TEST_ec_profile_warning: ceph osd erasure-code-profile set prof-shec_sse4 crush-failure-domain=osd plugin=shec_sse4 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: get_asok_path mon.a 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: CEPH_ARGS= 2026-02-19T13:09:47.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:111: TEST_ec_profile_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:47.749 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:47.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:112: TEST_ec_profile_warning: grep 'WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:47.758 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:47.574+0000 7f664a2f76c0 0 mon.a@0(leader).osd e24 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.758 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:47.685+0000 7f664a2f76c0 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.758 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:09:47.685+0000 7f664a2f76c0 0 mon.a@0(leader).osd e25 WARNING: erasure coding profile prof-shec_sse4 uses plugin shec_sse4 that has been deprecated. Please use shec instead. 2026-02-19T13:09:47.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:115: TEST_ec_profile_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:09:47.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:47.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:47.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:47.760 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:47.760 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:47.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:47.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:47.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:47.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:47.865 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:47.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:47.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:47.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:47.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:47.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:47.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:47.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:47.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:47.869 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:47.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:47.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:47.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:47.876 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:47.876 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.876 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:47.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:47.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning td/test-erasure-code-plugins 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:58: TEST_preload_no_warning: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:47.879 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:47.880 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:47.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:47.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:47.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:47.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:47.880 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:47.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:47.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:47.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:47.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:47.882 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:47.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:47.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:47.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:47.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:47.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:47.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:47.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:47.886 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:47.886 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.886 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:47.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:47.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:47.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:09:47.888 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:09:47.888 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.888 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.888 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:09:47.889 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:09:47.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:09:47.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:09:47.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:09:47.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure 2026-02-19T13:09:47.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:47.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:09:47.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:09:47.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:09:47.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:09:47.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure 2026-02-19T13:09:47.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:09:47.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:47.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:47.909 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:47.909 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.909 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:47.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure 2026-02-19T13:09:47.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:47.932 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.933 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.933 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:47.933 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:47.933 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:09:47.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:09:47.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:47.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:47.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:09:47.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:47.998 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:09:48.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:09:48.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:48.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:09:48.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:09:48.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:09:48.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:09:48.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:09:48.168 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:09:48.168 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:48.168 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:48.168 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:48.168 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:48.168 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:48.168 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:48.168 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:48.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:48.185 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-19T13:09:48.185 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:48.185 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:48.186 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:48.186 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:48.186 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:48.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:48.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T13:09:48.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:48.240 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:48.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure 2026-02-19T13:09:48.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:48.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:48.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:48.250 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:09:48.251 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:48.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0a32fa9b-94c9-432b-bf2f-c25228b43a3a 2026-02-19T13:09:48.252 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 0a32fa9b-94c9-432b-bf2f-c25228b43a3a 2026-02-19T13:09:48.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 0a32fa9b-94c9-432b-bf2f-c25228b43a3a' 2026-02-19T13:09:48.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:48.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCcC5dpooqlDxAADzyJ0XLj/JAn3zM3FRiYvQ== 2026-02-19T13:09:48.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCcC5dpooqlDxAADzyJ0XLj/JAn3zM3FRiYvQ=="}' 2026-02-19T13:09:48.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0a32fa9b-94c9-432b-bf2f-c25228b43a3a -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:48.368 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:48.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:48.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure --mkfs --key AQCcC5dpooqlDxAADzyJ0XLj/JAn3zM3FRiYvQ== --osd-uuid 0a32fa9b-94c9-432b-bf2f-c25228b43a3a 2026-02-19T13:09:48.394 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:48.396+0000 7f5778a70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:48.395 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:48.397+0000 7f5778a70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:48.396 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:48.398+0000 7f5778a70900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:48.397 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:48.399+0000 7f5778a70900 -1 bdev(0x55ee97a7ac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:48.397 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:48.399+0000 7f5778a70900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:09:48.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:09:48.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:48.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:09:48.879 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:09:48.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:49.180 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:09:49.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:09:49.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure 2026-02-19T13:09:49.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:49.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:49.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:49.194 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:49.196+0000 7f78945f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:49.195 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:49.197+0000 7f78945f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:49.196 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:49.198+0000 7f78945f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:49.371 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:49.372+0000 7f78945f9900 -1 Falling back to public interface 2026-02-19T13:09:49.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:09:49.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:49.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:09:49.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:49.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:49.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:49.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:49.403 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:49.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:49.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:49.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:49.772 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:49.773+0000 7f78945f9900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:09:50.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:50.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:50.620 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:09:50.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:50.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:50.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:50.841 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2771381723,v1:127.0.0.1:6803/2771381723] [v2:127.0.0.1:6804/2771381723,v1:127.0.0.1:6805/2771381723] exists,up 0a32fa9b-94c9-432b-bf2f-c25228b43a3a 2026-02-19T13:09:50.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:50.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:50.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:50.841 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-19T13:09:50.841 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:09:50.841 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:09:50.842 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:50.842 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:50.842 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:50.842 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:09:50.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T13:09:50.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:09:50.902 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:50.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:50.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:09:50.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:09:50.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:50.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:50.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:50.917 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:50.917 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:50.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:50.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:50.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:51.022 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:51.022 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:51.023 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:51.023 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:51.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:51.025 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:51.025 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:51.026 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:51.026 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:51.026 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:51.026 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:51.027 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:51.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:51.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:51.031 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:51.031 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.032 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:51.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:51.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:51.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:51.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:51.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:51.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:51.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:51.036 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:51.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:51.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:51.037 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:51.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:51.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:51.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:51.039 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:51.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:51.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:51.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:51.042 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:51.042 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.042 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.043 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:51.043 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:51.043 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:51.043 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:09:51.046 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:09:51.046 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.046 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:09:51.047 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:09:51.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:09:51.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:09:51.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:09:51.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec 2026-02-19T13:09:51.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:51.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:09:51.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:09:51.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:09:51.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:09:51.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec 2026-02-19T13:09:51.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:09:51.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:51.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:51.065 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:51.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.066 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:51.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec 2026-02-19T13:09:51.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:09:51.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:09:51.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:51.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:51.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:09:51.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:09:51.091 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:51.091 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:51.091 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:51.092 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:51.092 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.092 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.092 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:51.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:51.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:09:51.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:09:51.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:51.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:51.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:09:51.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:09:51.152 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:51.152 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:51.152 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:51.153 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:51.153 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.153 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.153 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:51.153 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:51.153 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:09:51.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:09:51.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:51.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:09:51.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:09:51.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:09:51.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:09:51.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:09:51.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:09:51.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:51.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:51.323 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:51.324 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.324 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:51.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:51.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:51.343 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-19T13:09:51.343 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:51.343 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:51.343 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:51.344 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.344 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.344 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:51.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T13:09:51.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:51.394 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:51.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:51.406 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:09:51.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:09:51.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:51.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:51.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:51.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:51.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:51.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec 2026-02-19T13:09:51.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:09:51.408 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:51.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d94f0cc5-db11-4611-8657-7176e36cf52d 2026-02-19T13:09:51.409 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 d94f0cc5-db11-4611-8657-7176e36cf52d 2026-02-19T13:09:51.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d94f0cc5-db11-4611-8657-7176e36cf52d' 2026-02-19T13:09:51.410 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:51.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCfC5dpmw7kGBAAe10vO8jUBwMCAMwJ3P1hGg== 2026-02-19T13:09:51.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCfC5dpmw7kGBAAe10vO8jUBwMCAMwJ3P1hGg=="}' 2026-02-19T13:09:51.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d94f0cc5-db11-4611-8657-7176e36cf52d -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:51.526 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:51.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:51.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec --mkfs --key AQCfC5dpmw7kGBAAe10vO8jUBwMCAMwJ3P1hGg== --osd-uuid d94f0cc5-db11-4611-8657-7176e36cf52d 2026-02-19T13:09:51.552 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:51.554+0000 7fa14ddce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:51.553 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:51.555+0000 7fa14ddce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:51.554 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:51.556+0000 7fa14ddce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:51.555 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:51.556+0000 7fa14ddce900 -1 bdev(0x556c29926c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:51.555 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:51.556+0000 7fa14ddce900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:09:52.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:09:52.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:52.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:09:52.003 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:09:52.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:52.302 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:09:52.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:09:52.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec 2026-02-19T13:09:52.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:52.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:52.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:52.316 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:52.318+0000 7fd0664ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:52.318 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:52.320+0000 7fd0664ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:52.319 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:52.321+0000 7fd0664ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:52.514 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:52.515+0000 7fd0664ad900 -1 Falling back to public interface 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:52.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:52.889 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:52.891+0000 7fd0664ad900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:09:53.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:53.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:53.752 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:09:53.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:53.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:53.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:53.973 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/934955033,v1:127.0.0.1:6803/934955033] [v2:127.0.0.1:6804/934955033,v1:127.0.0.1:6805/934955033] exists,up d94f0cc5-db11-4611-8657-7176e36cf52d 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:53.975 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:09:53.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T13:09:53.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:09:54.034 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:54.041 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:54.043 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:09:54.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:09:54.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:54.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:54.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:54.045 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:54.045 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:54.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:54.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:54.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:54.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:54.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:54.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:54.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:54.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:54.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:54.153 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:54.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:54.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:54.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:54.155 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:54.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:54.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:54.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:54.159 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:54.160 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.160 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:54.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:54.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:54.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-19T13:09:54.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-19T13:09:54.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:54.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:54.164 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:54.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:54.167 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:54.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:54.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:54.167 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:54.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:54.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:54.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:54.168 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:54.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:54.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:54.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:54.169 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:54.169 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.169 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:54.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:54.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:54.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:09:54.171 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:09:54.171 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.171 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=lrc 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:09:54.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:09:54.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:09:54.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=lrc 2026-02-19T13:09:54.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:09:54.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:54.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:54.192 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:54.192 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.192 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:54.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=lrc 2026-02-19T13:09:54.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:09:54.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:09:54.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:54.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:54.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:09:54.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:09:54.217 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:54.217 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:54.217 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:54.218 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:54.218 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.218 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.218 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:54.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:54.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:09:54.279 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:09:54.279 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.280 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:54.281 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:54.281 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:09:54.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:09:54.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:54.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:09:54.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:09:54.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:09:54.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:09:54.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:09:54.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:09:54.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:54.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:54.452 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:54.452 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.452 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:54.453 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:54.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:54.468 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-19T13:09:54.468 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:54.468 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:54.468 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:54.468 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.468 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.468 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:54.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T13:09:54.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:54.522 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:54.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=lrc 2026-02-19T13:09:54.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:54.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:54.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:09:54.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:54.530 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=lrc 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:09:54.531 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:54.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3e852d19-b308-40db-a704-f3bf37ab7819 2026-02-19T13:09:54.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 3e852d19-b308-40db-a704-f3bf37ab7819' 2026-02-19T13:09:54.533 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 3e852d19-b308-40db-a704-f3bf37ab7819 2026-02-19T13:09:54.534 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:54.544 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCiC5dpaaOLIBAALsr23/VniFpHpb/0wmBl4Q== 2026-02-19T13:09:54.544 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCiC5dpaaOLIBAALsr23/VniFpHpb/0wmBl4Q=="}' 2026-02-19T13:09:54.544 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3e852d19-b308-40db-a704-f3bf37ab7819 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:54.651 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:54.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:54.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc --mkfs --key AQCiC5dpaaOLIBAALsr23/VniFpHpb/0wmBl4Q== --osd-uuid 3e852d19-b308-40db-a704-f3bf37ab7819 2026-02-19T13:09:54.678 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:54.680+0000 7fd11e361900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:54.679 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:54.681+0000 7fd11e361900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:54.680 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:54.682+0000 7fd11e361900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:54.680 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:54.682+0000 7fd11e361900 -1 bdev(0x563bd74d4c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:54.680 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:54.682+0000 7fd11e361900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:09:55.113 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:09:55.114 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:55.114 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:09:55.114 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:09:55.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:55.433 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:09:55.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:09:55.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=lrc 2026-02-19T13:09:55.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:55.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:55.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:55.447 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:55.449+0000 7ff67f3e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:55.448 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:55.450+0000 7ff67f3e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:55.449 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:55.451+0000 7ff67f3e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:55.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:55.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:55.659 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:55.661+0000 7ff67f3e7900 -1 Falling back to public interface 2026-02-19T13:09:55.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:55.916 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:55.917+0000 7ff67f3e7900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:09:56.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:56.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:56.874 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:09:56.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:56.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:56.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:57.099 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1259347771,v1:127.0.0.1:6803/1259347771] [v2:127.0.0.1:6804/1259347771,v1:127.0.0.1:6805/1259347771] exists,up 3e852d19-b308-40db-a704-f3bf37ab7819 2026-02-19T13:09:57.099 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:09:57.099 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:09:57.099 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T13:09:57.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:09:57.159 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:57.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:09:57.166 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:09:57.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:09:57.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:57.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:57.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:57.169 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:57.169 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:57.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:57.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:57.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:57.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:57.275 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:57.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:57.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:57.277 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:57.277 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:57.277 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:57.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:57.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:57.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:57.279 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:57.280 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:57.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:57.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:57.285 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:57.285 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.285 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:57.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:57.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:60: TEST_preload_no_warning: for plugin in ${plugins[*]} 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:61: TEST_preload_no_warning: setup td/test-erasure-code-plugins 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:09:57.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:09:57.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:09:57.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:09:57.290 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:09:57.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:09:57.290 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:09:57.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:09:57.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:09:57.291 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:09:57.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:57.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:09:57.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:09:57.292 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:09:57.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:09:57.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:09:57.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:09:57.295 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:09:57.295 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.295 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:09:57.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:09:57.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:09:57.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:09:57.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:09:57.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:62: TEST_preload_no_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=isa 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:09:57.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=isa 2026-02-19T13:09:57.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:09:57.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:57.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:57.317 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:57.317 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.317 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:57.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=isa 2026-02-19T13:09:57.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:09:57.340 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:09:57.340 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:57.340 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:57.341 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:09:57.341 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:09:57.342 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:57.342 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:57.342 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:57.342 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:57.342 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.342 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.342 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:57.343 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:57.343 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:09:57.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:09:57.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:09:57.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:09:57.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:09:57.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:09:57.406 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:09:57.406 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:57.406 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:57.406 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:57.406 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.407 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.407 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:57.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:09:57.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:09:57.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:63: TEST_preload_no_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:09:57.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:57.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:09:57.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:09:57.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:09:57.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:09:57.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:09:57.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:09:57.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:57.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:57.576 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:57.576 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.576 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:57.577 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:57.577 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:09:57.594 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: get_asok_path mon.a 2026-02-19T13:09:57.594 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:09:57.594 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:09:57.594 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:09:57.594 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.595 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.595 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:09:57.595 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T13:09:57.595 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:64: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:09:57.646 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:65: TEST_preload_no_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=isa 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:09:57.654 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=isa 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:09:57.655 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:09:57.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=68b8989e-37e0-4c1f-b8a0-58fcc52d39ab 2026-02-19T13:09:57.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 68b8989e-37e0-4c1f-b8a0-58fcc52d39ab' 2026-02-19T13:09:57.656 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 68b8989e-37e0-4c1f-b8a0-58fcc52d39ab 2026-02-19T13:09:57.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:09:57.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQClC5dpi3rkJxAAQqm91HzTmPgaofC7K/c43w== 2026-02-19T13:09:57.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQClC5dpi3rkJxAAQqm91HzTmPgaofC7K/c43w=="}' 2026-02-19T13:09:57.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 68b8989e-37e0-4c1f-b8a0-58fcc52d39ab -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:57.776 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:57.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:09:57.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa --mkfs --key AQClC5dpi3rkJxAAQqm91HzTmPgaofC7K/c43w== --osd-uuid 68b8989e-37e0-4c1f-b8a0-58fcc52d39ab 2026-02-19T13:09:57.801 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:57.803+0000 7f6e85dd3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:57.802 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:57.804+0000 7f6e85dd3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:57.803 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:57.805+0000 7f6e85dd3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:57.804 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:57.805+0000 7f6e85dd3900 -1 bdev(0x55eea8f88c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:09:57.804 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:57.805+0000 7f6e85dd3900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:09:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:09:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:09:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:09:58.254 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:09:58.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:09:58.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:09:58.550 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:09:58.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=isa 2026-02-19T13:09:58.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:09:58.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:09:58.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:09:58.564 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:58.566+0000 7fcf48737900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:58.565 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:58.567+0000 7fcf48737900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:58.566 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:58.568+0000 7fcf48737900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:09:58.757 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:58.759+0000 7fcf48737900 -1 Falling back to public interface 2026-02-19T13:09:58.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:09:58.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:09:58.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:09:58.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:09:58.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:09:58.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:58.772 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:09:58.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:09:58.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:58.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:09:58.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:09:59.092 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:09:59.093+0000 7fcf48737900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:09:59.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:09:59.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:09:59.994 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:09:59.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:09:59.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:09:59.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:00.213 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1822416084,v1:127.0.0.1:6803/1822416084] [v2:127.0.0.1:6804/1822416084,v1:127.0.0.1:6805/1822416084] exists,up 68b8989e-37e0-4c1f-b8a0-58fcc52d39ab 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: get_asok_path osd.0 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.214 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:10:00.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: CEPH_ARGS= 2026-02-19T13:10:00.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:66: TEST_preload_no_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:10:00.271 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:00.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:67: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:10:00.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:68: TEST_preload_no_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:10:00.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:69: TEST_preload_no_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:10:00.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:00.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:00.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:00.282 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:00.282 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:00.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:00.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:00.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:00.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:00.387 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:00.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:00.388 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:00.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:00.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:00.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:00.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:00.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:00.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:00.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:00.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:00.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:00.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:00.395 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:00.395 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.395 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:00.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:72: TEST_preload_no_warning: return 0 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_no_warning_default td/test-erasure-code-plugins 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:76: TEST_preload_no_warning_default: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:78: TEST_preload_no_warning_default: setup td/test-erasure-code-plugins 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:00.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:00.399 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:00.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:00.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:00.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:00.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:00.400 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:00.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:00.400 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:00.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:00.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:00.401 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:00.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:00.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:00.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:00.402 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:00.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:00.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:00.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:00.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:00.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:00.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:00.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:00.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:10:00.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:00.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.408 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:10:00.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:00.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:00.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:00.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:10:00.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:79: TEST_preload_no_warning_default: run_mon td/test-erasure-code-plugins a 2026-02-19T13:10:00.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:00.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:00.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:00.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:00.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:10:00.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins 2026-02-19T13:10:00.428 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:00.428 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:00.428 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:00.428 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:00.429 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.429 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.429 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:00.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:10:00.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:00.454 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.455 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:00.456 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:00.456 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:00.515 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.516 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.516 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:00.516 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:00.516 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:00.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: get_asok_path mon.a 2026-02-19T13:10:00.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:00.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:00.575 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:00.575 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.575 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:00.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-19T13:10:00.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:80: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:10:00.630 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:00.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:81: TEST_preload_no_warning_default: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:10:00.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:00.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:00.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:00.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:00.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:10:00.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:00.748 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:00.748 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:00.748 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:00.749 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:00.749 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.749 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.749 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:00.749 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:00.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:82: TEST_preload_no_warning_default: run_osd td/test-erasure-code-plugins 0 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:00.767 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:10:00.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:00.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=385956cb-e860-4b4b-a86c-deb56ac5b04b 2026-02-19T13:10:00.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 385956cb-e860-4b4b-a86c-deb56ac5b04b' 2026-02-19T13:10:00.769 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 385956cb-e860-4b4b-a86c-deb56ac5b04b 2026-02-19T13:10:00.769 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:00.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCoC5dpWX6gLhAAlUl9QIn9oafvz1TQKIP2cg== 2026-02-19T13:10:00.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCoC5dpWX6gLhAAlUl9QIn9oafvz1TQKIP2cg=="}' 2026-02-19T13:10:00.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 385956cb-e860-4b4b-a86c-deb56ac5b04b -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:00.889 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:00.899 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:00.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCoC5dpWX6gLhAAlUl9QIn9oafvz1TQKIP2cg== --osd-uuid 385956cb-e860-4b4b-a86c-deb56ac5b04b 2026-02-19T13:10:00.915 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:00.917+0000 7fa276595900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:00.916 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:00.918+0000 7fa276595900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:00.917 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:00.919+0000 7fa276595900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:00.917 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:00.919+0000 7fa276595900 -1 bdev(0x55f9dea7ac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:00.917 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:00.919+0000 7fa276595900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:10:01.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:10:01.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:01.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:01.367 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:10:01.368 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:01.664 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:10:01.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:01.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:01.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:01.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:01.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:01.678 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:01.680+0000 7f11e3f37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:01.679 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:01.681+0000 7f11e3f37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:01.680 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:01.682+0000 7f11e3f37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:01.813 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:01.814+0000 7f11e3f37900 -1 Falling back to public interface 2026-02-19T13:10:01.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:01.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:01.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:01.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:01.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:01.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:01.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:01.887 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:01.888 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:01.888 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:02.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:02.214 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:02.216+0000 7f11e3f37900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:03.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:03.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:03.104 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:03.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:03.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:03.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:03.322 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2676881328,v1:127.0.0.1:6803/2676881328] [v2:127.0.0.1:6804/2676881328,v1:127.0.0.1:6805/2676881328] exists,up 385956cb-e860-4b4b-a86c-deb56ac5b04b 2026-02-19T13:10:03.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:03.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:03.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:03.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: get_asok_path osd.0 2026-02-19T13:10:03.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:03.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:03.324 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:03.324 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.324 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:10:03.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: CEPH_ARGS= 2026-02-19T13:10:03.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:83: TEST_preload_no_warning_default: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:10:03.383 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:03.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:84: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:10:03.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:85: TEST_preload_no_warning_default: grep 'WARNING: osd_erasure_code_plugins' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:10:03.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:86: TEST_preload_no_warning_default: teardown td/test-erasure-code-plugins 2026-02-19T13:10:03.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:03.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:03.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:03.393 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:03.393 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:03.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:03.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:03.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:03.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:03.500 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:03.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:03.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:03.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:03.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:03.502 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:03.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:03.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:03.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:03.503 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:03.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:03.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:03.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:03.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:03.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:88: TEST_preload_no_warning_default: return 0 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:35: run: for func in $funcs 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:36: run: TEST_preload_warning td/test-erasure-code-plugins 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:41: TEST_preload_warning: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:03.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:03.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:03.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:03.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:03.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:03.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:03.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:03.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:03.511 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:03.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:03.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:03.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:03.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:03.514 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:03.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:03.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:03.515 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:03.515 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:03.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:03.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:03.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:03.518 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:03.518 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.518 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:03.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:03.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:03.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:10:03.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:03.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:10:03.521 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_generic 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:10:03.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_generic 2026-02-19T13:10:03.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:03.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:03.542 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:03.542 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:03.542 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.542 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.542 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:03.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_generic 2026-02-19T13:10:03.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:03.565 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:03.565 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:03.565 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:03.565 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:03.565 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:03.565 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:03.565 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:03.565 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:03.566 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:03.566 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.566 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.566 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:03.566 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:03.566 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:10:03.626 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:03.626 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:03.626 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:03.626 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:03.627 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:03.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:10:03.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:03.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:03.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:03.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:03.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:10:03.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:03.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:03.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:03.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:03.794 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:03.794 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.794 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:03.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:03.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:03.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:10:03.865 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_generic 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_generic 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:10:03.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:03.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b9e1982a-a408-4cae-9429-06e00b56ba84 2026-02-19T13:10:03.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 b9e1982a-a408-4cae-9429-06e00b56ba84' 2026-02-19T13:10:03.876 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 b9e1982a-a408-4cae-9429-06e00b56ba84 2026-02-19T13:10:03.876 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:03.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCrC5dpCMgBNRAAYCwcWPNihc1ZZ0vKC2+4YQ== 2026-02-19T13:10:03.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCrC5dpCMgBNRAAYCwcWPNihc1ZZ0vKC2+4YQ=="}' 2026-02-19T13:10:03.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b9e1982a-a408-4cae-9429-06e00b56ba84 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:03.997 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:04.008 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:04.008 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic --mkfs --key AQCrC5dpCMgBNRAAYCwcWPNihc1ZZ0vKC2+4YQ== --osd-uuid b9e1982a-a408-4cae-9429-06e00b56ba84 2026-02-19T13:10:04.023 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.025+0000 7fe7ba933900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:04.024 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.026+0000 7fe7ba933900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:04.025 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.027+0000 7fe7ba933900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:04.026 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.027+0000 7fe7ba933900 -1 bdev(0x55efe9700c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:04.026 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.027+0000 7fe7ba933900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:10:04.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:10:04.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:04.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:04.452 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:10:04.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:04.750 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:10:04.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:04.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_generic 2026-02-19T13:10:04.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:04.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:04.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:04.765 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.766+0000 7fc2baae0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:04.766 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.768+0000 7fc2baae0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:04.767 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.769+0000 7fc2baae0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:04.958 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:04.960+0000 7fc2baae0900 -1 Falling back to public interface 2026-02-19T13:10:04.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:04.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:05.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:05.297 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:05.299+0000 7fc2baae0900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:06.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:06.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:06.188 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:06.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:06.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:06.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:06.409 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3984970048,v1:127.0.0.1:6803/3984970048] [v2:127.0.0.1:6804/3984970048,v1:127.0.0.1:6805/3984970048] exists,up b9e1982a-a408-4cae-9429-06e00b56ba84 2026-02-19T13:10:06.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:06.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:06.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:06.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T13:10:06.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:06.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:06.410 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:06.410 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.410 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.410 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:10:06.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:06.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:10:06.469 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:06.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:10:06.478 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:03.558+0000 7fd144ca1dc0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T13:10:06.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:10:06.479 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:04.961+0000 7fc2baae0900 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T13:10:06.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:10:06.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:06.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:06.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:06.480 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:06.480 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:06.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:06.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:06.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:06.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:06.587 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:06.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:06.588 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:06.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:06.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:06.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:06.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:06.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:06.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:06.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:06.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:06.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:06.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:06.596 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:06.596 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.596 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:06.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:06.598 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:06.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:06.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:06.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:06.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:06.599 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:06.600 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:06.600 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:06.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:06.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:06.602 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:06.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:06.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:06.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:06.603 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:06.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:06.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:06.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:06.605 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:06.605 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.605 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:06.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:06.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:06.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:10:06.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:06.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.608 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:10:06.609 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:06.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:06.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:06.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:10:06.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T13:10:06.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:06.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:06.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:06.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:06.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:10:06.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T13:10:06.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:06.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:06.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:06.628 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:06.628 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.628 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:06.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T13:10:06.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:06.651 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:06.651 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:06.651 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:06.651 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:06.652 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:06.652 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:06.652 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:06.652 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:06.652 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:06.652 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.652 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.652 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:06.653 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:06.653 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:10:06.712 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:06.712 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:06.712 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:06.712 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:06.712 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:06.712 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:06.712 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:06.712 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:06.713 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:06.713 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.713 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.713 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:06.713 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:06.713 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:06.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:10:06.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:06.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:06.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:06.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:06.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:10:06.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:06.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:06.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:06.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:06.884 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:06.884 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.884 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:06.885 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:06.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:06.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:10:06.955 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:06.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T13:10:06.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:06.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:06.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:06.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:06.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse3 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=62133a60-7ed3-40a0-95e7-2650d5317f79 2026-02-19T13:10:06.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 62133a60-7ed3-40a0-95e7-2650d5317f79' 2026-02-19T13:10:06.970 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 62133a60-7ed3-40a0-95e7-2650d5317f79 2026-02-19T13:10:06.970 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:06.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCuC5dpmKRTOhAATYtyEug1ngRnEv13A6kOaw== 2026-02-19T13:10:06.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCuC5dpmKRTOhAATYtyEug1ngRnEv13A6kOaw=="}' 2026-02-19T13:10:06.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 62133a60-7ed3-40a0-95e7-2650d5317f79 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:07.085 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:07.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:07.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 --mkfs --key AQCuC5dpmKRTOhAATYtyEug1ngRnEv13A6kOaw== --osd-uuid 62133a60-7ed3-40a0-95e7-2650d5317f79 2026-02-19T13:10:07.111 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:07.113+0000 7f226f3e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:07.112 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:07.114+0000 7f226f3e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:07.113 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:07.115+0000 7f226f3e7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:07.113 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:07.116+0000 7f226f3e7900 -1 bdev(0x5628a1db8c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:07.114 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:07.116+0000 7f226f3e7900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:10:07.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:10:07.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:07.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:07.550 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:10:07.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:07.846 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:10:07.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse3 2026-02-19T13:10:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:07.861 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:07.863+0000 7f4792564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:07.863 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:07.865+0000 7f4792564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:07.864 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:07.866+0000 7f4792564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:08.044 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:08.046+0000 7f4792564900 -1 Falling back to public interface 2026-02-19T13:10:08.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:08.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:08.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:08.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:08.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:08.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:08.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:08.066 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:08.067 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:08.067 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:08.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:08.440 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:08.442+0000 7f4792564900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:09.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:09.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:09.288 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:09.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:09.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:09.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:09.551 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2895919408,v1:127.0.0.1:6803/2895919408] [v2:127.0.0.1:6804/2895919408,v1:127.0.0.1:6805/2895919408] exists,up 62133a60-7ed3-40a0-95e7-2650d5317f79 2026-02-19T13:10:09.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:09.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:09.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:09.553 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T13:10:09.553 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:09.553 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:09.554 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:09.554 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:09.554 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:09.554 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:10:09.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:09.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:10:09.609 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:09.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:10:09.617 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:06.645+0000 7f606f6cddc0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T13:10:09.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:10:09.618 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:08.047+0000 7f4792564900 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T13:10:09.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:10:09.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:09.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:09.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:09.620 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:09.620 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:09.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:09.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:09.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:09.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:09.725 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:09.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:09.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:09.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:09.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:09.727 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:09.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:09.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:09.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:09.729 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:09.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:09.730 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:09.730 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:09.733 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:09.733 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:09.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:09.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:09.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:09.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:09.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:09.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:09.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:09.737 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:09.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:09.738 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:09.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:09.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:09.739 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:09.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:09.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:09.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:09.741 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:09.741 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:09.742 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:09.742 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:09.743 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:09.743 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:09.743 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:09.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:09.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:09.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:09.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:10:09.745 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:09.745 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:09.746 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:09.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:10:09.746 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:09.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:10:09.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T13:10:09.766 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:09.766 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:09.766 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:09.766 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:09.766 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:09.766 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:09.766 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:09.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T13:10:09.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:09.789 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:09.789 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:09.789 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:09.789 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:09.789 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:09.789 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:09.789 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:09.790 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:09.790 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:09.790 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:09.790 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:09.790 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:09.790 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:09.790 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:10:09.850 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:09.851 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:09.852 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:09.852 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:09.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:09.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:09.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:10:09.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:09.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:09.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:09.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:09.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:10:09.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:10.016 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:10.016 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:10.016 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:10.016 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:10.016 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:10.016 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:10.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:10.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:10.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:10.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T13:10:10.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:10.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:10.035 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:10.035 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:10.035 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:10.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:10.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:10.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:10:10.090 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:10.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T13:10:10.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:10.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:10.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:10.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:10.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:10.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=jerasure_sse4 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2027aa5b-0591-4ca7-881f-62d5825911a4 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 2027aa5b-0591-4ca7-881f-62d5825911a4' 2026-02-19T13:10:10.104 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 2027aa5b-0591-4ca7-881f-62d5825911a4 2026-02-19T13:10:10.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:10.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCyC5dpOcTABhAAYDT3L6UCAZLQNAiOAUa7oQ== 2026-02-19T13:10:10.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCyC5dpOcTABhAAYDT3L6UCAZLQNAiOAUa7oQ=="}' 2026-02-19T13:10:10.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2027aa5b-0591-4ca7-881f-62d5825911a4 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:10.219 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:10.229 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:10.229 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 --mkfs --key AQCyC5dpOcTABhAAYDT3L6UCAZLQNAiOAUa7oQ== --osd-uuid 2027aa5b-0591-4ca7-881f-62d5825911a4 2026-02-19T13:10:10.245 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:10.246+0000 7ffafa57d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:10.246 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:10.248+0000 7ffafa57d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:10.247 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:10.249+0000 7ffafa57d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:10.247 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:10.249+0000 7ffafa57d900 -1 bdev(0x557da79dac00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:10.247 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:10.249+0000 7ffafa57d900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:10:10.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:10:10.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:10.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:10.669 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:10:10.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:10.966 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:10:10.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:10.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=jerasure_sse4 2026-02-19T13:10:10.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:10.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:10.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:10.979 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:10.981+0000 7f5857d56900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:10.981 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:10.983+0000 7f5857d56900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:10.982 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:10.984+0000 7f5857d56900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:11.181 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:11.184+0000 7f5857d56900 -1 Falling back to public interface 2026-02-19T13:10:11.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:11.565 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:11.567+0000 7f5857d56900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:12.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:12.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:12.404 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:12.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:12.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:12.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:12.621 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3809548902,v1:127.0.0.1:6803/3809548902] [v2:127.0.0.1:6804/3809548902,v1:127.0.0.1:6805/3809548902] exists,up 2027aa5b-0591-4ca7-881f-62d5825911a4 2026-02-19T13:10:12.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:12.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:12.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:12.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T13:10:12.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:12.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:12.623 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:12.624 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:12.624 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:12.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:10:12.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:12.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:10:12.679 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:12.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:10:12.687 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:09.782+0000 7fdcf2173dc0 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T13:10:12.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:10:12.688 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:11.185+0000 7f5857d56900 0 WARNING: osd_erasure_code_plugins contains plugin jerasure_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use jerasure instead. 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:12.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:12.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:12.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:12.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:12.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:12.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:12.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:12.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:12.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:12.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:12.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:12.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:12.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:12.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:12.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:12.804 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:12.804 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:12.804 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:12.804 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:12.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:12.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:12.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T13:10:12.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T13:10:12.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:12.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:10:12.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:12.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:12.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:12.807 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:12.807 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:12.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:12.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:12.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:12.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:12.808 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:12.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:12.808 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:12.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:12.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:12.810 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:12.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:12.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:12.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:12.811 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:12.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:12.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:12.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:12.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:12.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:12.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:12.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:12.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:12.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:12.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:10:12.816 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:12.816 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:12.816 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:12.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:10:12.817 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:12.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_generic 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:10:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_generic 2026-02-19T13:10:12.837 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:12.837 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:12.838 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:12.838 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:12.838 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:12.838 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:12.838 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:12.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_generic 2026-02-19T13:10:12.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:12.865 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:12.865 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:12.865 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:12.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:10:12.924 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:12.925 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:12.926 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:12.926 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:12.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:10:12.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:12.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:12.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:12.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:12.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:10:12.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:13.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:13.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:13.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:13.101 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:13.102 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:13.102 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:13.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:13.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:13.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:13.120 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T13:10:13.121 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:13.121 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:13.121 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:13.121 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:13.121 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:13.121 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:13.121 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:13.121 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:10:13.173 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:13.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_generic 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:13.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_generic 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:10:13.187 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:13.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=77d9f54d-70a8-49e3-bb05-824df59429b1 2026-02-19T13:10:13.188 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 77d9f54d-70a8-49e3-bb05-824df59429b1 2026-02-19T13:10:13.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 77d9f54d-70a8-49e3-bb05-824df59429b1' 2026-02-19T13:10:13.189 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:13.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC1C5dpwrTPCxAAuAv5sgacV6qNmSOQJIGPPw== 2026-02-19T13:10:13.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC1C5dpwrTPCxAAuAv5sgacV6qNmSOQJIGPPw=="}' 2026-02-19T13:10:13.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 77d9f54d-70a8-49e3-bb05-824df59429b1 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:13.303 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:13.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:13.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic --mkfs --key AQC1C5dpwrTPCxAAuAv5sgacV6qNmSOQJIGPPw== --osd-uuid 77d9f54d-70a8-49e3-bb05-824df59429b1 2026-02-19T13:10:13.328 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:13.330+0000 7f8b5d612900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:13.330 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:13.332+0000 7f8b5d612900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:13.331 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:13.333+0000 7f8b5d612900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:13.331 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:13.333+0000 7f8b5d612900 -1 bdev(0x55c39d7dcc00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:13.331 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:13.333+0000 7f8b5d612900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:10:13.804 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:10:13.805 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:13.805 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:13.806 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:10:13.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:14.114 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:10:14.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:14.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_generic 2026-02-19T13:10:14.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:14.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:14.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:14.128 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:14.130+0000 7f5f16e81900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:14.130 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:14.132+0000 7f5f16e81900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:14.131 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:14.133+0000 7f5f16e81900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:14.334 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:14.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:14.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:14.362 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:14.364+0000 7f5f16e81900 -1 Falling back to public interface 2026-02-19T13:10:14.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:14.688 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:14.690+0000 7f5f16e81900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:15.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:15.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:15.553 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:15.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:15.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:15.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:15.776 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1481099274,v1:127.0.0.1:6803/1481099274] [v2:127.0.0.1:6804/1481099274,v1:127.0.0.1:6805/1481099274] exists,up 77d9f54d-70a8-49e3-bb05-824df59429b1 2026-02-19T13:10:15.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:15.778 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:10:15.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:15.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:10:15.837 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:15.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:10:15.845 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:12.855+0000 7f27818d7dc0 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T13:10:15.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_generic' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:14.365+0000 7f5f16e81900 0 WARNING: osd_erasure_code_plugins contains plugin shec_generic that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:15.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:15.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:15.954 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:15.955 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:15.955 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:15.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:15.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:15.956 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:15.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:15.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:15.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:15.957 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:15.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:15.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:15.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:15.962 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:15.962 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:15.962 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:15.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:15.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:15.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:15.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T13:10:15.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T13:10:15.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:15.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:10:15.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:15.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:15.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:15.965 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:15.965 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:15.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:15.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:15.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:15.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:15.966 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:15.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:15.966 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:15.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:15.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:15.968 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:15.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:15.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:15.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:15.969 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:15.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:15.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:15.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:15.972 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:15.972 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:15.972 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:15.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:15.973 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:15.973 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:15.973 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:10:15.974 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:15.974 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:15.975 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:15.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:10:15.975 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse3 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:10:15.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse3 2026-02-19T13:10:15.996 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:15.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:15.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:15.997 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:15.997 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:15.997 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:15.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:15.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse3 2026-02-19T13:10:16.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:16.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:16.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:16.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:16.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:16.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:16.022 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:16.022 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:16.022 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:16.023 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:16.023 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:16.023 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:16.023 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:16.023 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:16.023 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:10:16.083 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:16.083 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:16.084 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:16.084 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:16.084 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:16.085 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:16.085 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:16.085 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:16.085 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:16.085 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:16.085 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:16.086 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:16.086 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:16.086 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:16.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:10:16.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:16.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:16.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:16.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:16.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:10:16.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:16.260 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:16.260 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:16.260 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:16.260 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:16.260 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:16.260 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:16.260 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:16.261 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:16.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:16.278 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T13:10:16.278 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:16.278 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:16.279 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:16.279 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:16.279 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:16.279 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:16.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:16.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:10:16.333 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse3 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:16.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:16.345 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:16.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:16.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse3 2026-02-19T13:10:16.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:10:16.347 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:16.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=57a38e62-f1e1-4df5-b921-65fc53fe63de 2026-02-19T13:10:16.347 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 57a38e62-f1e1-4df5-b921-65fc53fe63de 2026-02-19T13:10:16.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 57a38e62-f1e1-4df5-b921-65fc53fe63de' 2026-02-19T13:10:16.348 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:16.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC4C5dpr/I4FRAAa880+0PkuOBiruUC4N7ljQ== 2026-02-19T13:10:16.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC4C5dpr/I4FRAAa880+0PkuOBiruUC4N7ljQ=="}' 2026-02-19T13:10:16.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 57a38e62-f1e1-4df5-b921-65fc53fe63de -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:16.462 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:16.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:16.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 --mkfs --key AQC4C5dpr/I4FRAAa880+0PkuOBiruUC4N7ljQ== --osd-uuid 57a38e62-f1e1-4df5-b921-65fc53fe63de 2026-02-19T13:10:16.489 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:16.491+0000 7fde91dd3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:16.490 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:16.492+0000 7fde91dd3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:16.491 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:16.493+0000 7fde91dd3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:16.492 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:16.493+0000 7fde91dd3900 -1 bdev(0x5654740e6c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:16.492 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:16.493+0000 7fde91dd3900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:10:16.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:10:16.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:16.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:16.937 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:10:16.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:17.242 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:10:17.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:17.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse3 2026-02-19T13:10:17.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:17.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:17.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:17.257 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:17.259+0000 7f9160c5a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:17.258 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:17.260+0000 7f9160c5a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:17.259 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:17.261+0000 7f9160c5a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:17.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:17.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:17.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:17.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:17.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:17.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:17.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:17.465 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:17.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:17.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:17.467 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:17.469+0000 7f9160c5a900 -1 Falling back to public interface 2026-02-19T13:10:17.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:17.841 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:17.843+0000 7f9160c5a900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:18.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:18.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:18.683 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:18.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:18.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:18.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:18.908 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2857411710,v1:127.0.0.1:6803/2857411710] [v2:127.0.0.1:6804/2857411710,v1:127.0.0.1:6805/2857411710] exists,up 57a38e62-f1e1-4df5-b921-65fc53fe63de 2026-02-19T13:10:18.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:18.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:18.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:18.910 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T13:10:18.910 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:18.910 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:18.910 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:18.911 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:18.911 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:18.911 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:10:18.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:18.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:10:18.969 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:18.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:10:18.979 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:16.013+0000 7fc3f2ddfdc0 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T13:10:18.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse3' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:10:18.981 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:17.469+0000 7f9160c5a900 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse3 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:18.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:19.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:19.088 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:19.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:19.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:19.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:19.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:19.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:19.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:19.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:19.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:19.093 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:19.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:19.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:19.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:19.098 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:19.098 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.098 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:43: TEST_preload_warning: for plugin in ${legacy_jerasure_plugins[*]} ${legacy_shec_plugins[*]} 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:44: TEST_preload_warning: setup td/test-erasure-code-plugins 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code-plugins 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:19.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:19.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:19.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:19.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:19.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:19.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:19.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:19.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:19.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:19.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:19.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:19.103 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:19.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:19.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:19.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:19.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:19.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:19.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:19.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:19.107 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:19.107 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.107 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:19.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:19.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:19.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code-plugins 2026-02-19T13:10:19.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:19.110 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.110 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.12230 2026-02-19T13:10:19.111 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code-plugins 1' TERM HUP INT 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:45: TEST_preload_warning: run_mon td/test-erasure-code-plugins a --osd_erasure_code_plugins=shec_sse4 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:19.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code-plugins/a 2026-02-19T13:10:19.113 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code-plugins/a --run-dir=td/test-erasure-code-plugins --osd_erasure_code_plugins=shec_sse4 2026-02-19T13:10:19.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:19.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:19.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:19.131 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:19.131 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.131 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:19.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code-plugins/a '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code-plugins/log --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false --osd_erasure_code_plugins=shec_sse4 2026-02-19T13:10:19.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:19.156 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.157 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:19.158 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:19.158 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get fsid 2026-02-19T13:10:19.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:19.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:19.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:19.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:19.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.12230/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:19.277 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:46: TEST_preload_warning: run_mgr td/test-erasure-code-plugins x 2026-02-19T13:10:19.277 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:19.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:19.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:19.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:19.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code-plugins/x 2026-02-19T13:10:19.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:19.390 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:19.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:19.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:19.391 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:19.391 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.391 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:19.392 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:19.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code-plugins/x '--log-file=td/test-erasure-code-plugins/$name.log' '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --run-dir=td/test-erasure-code-plugins '--pid-file=td/test-erasure-code-plugins/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:19.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: get_asok_path mon.a 2026-02-19T13:10:19.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:19.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:19.409 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:19.409 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.410 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.410 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-mon.a.asok 2026-02-19T13:10:19.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:19.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:47: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-mon.a.asok log flush 2026-02-19T13:10:19.461 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:48: TEST_preload_warning: run_osd td/test-erasure-code-plugins 0 --osd_erasure_code_plugins=shec_sse4 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code-plugins/0 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 ' 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:19.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code-plugins/0' 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code-plugins/0/journal' 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code-plugins' 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:19.472 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code-plugins/$name.log' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code-plugins/$name.pid' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:19.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:19.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+=--osd_erasure_code_plugins=shec_sse4 2026-02-19T13:10:19.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code-plugins/0 2026-02-19T13:10:19.474 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:19.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1d08ddc0-5865-4290-8813-c729cf0f74c1 2026-02-19T13:10:19.474 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 1d08ddc0-5865-4290-8813-c729cf0f74c1 2026-02-19T13:10:19.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 1d08ddc0-5865-4290-8813-c729cf0f74c1' 2026-02-19T13:10:19.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:19.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC7C5dpp0LYHBAAIBxfp+1TYWvytGN/YT43Xg== 2026-02-19T13:10:19.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC7C5dpp0LYHBAAIBxfp+1TYWvytGN/YT43Xg=="}' 2026-02-19T13:10:19.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1d08ddc0-5865-4290-8813-c729cf0f74c1 -i td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:19.592 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:19.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code-plugins/0/new.json 2026-02-19T13:10:19.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 --mkfs --key AQC7C5dpp0LYHBAAIBxfp+1TYWvytGN/YT43Xg== --osd-uuid 1d08ddc0-5865-4290-8813-c729cf0f74c1 2026-02-19T13:10:19.618 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:19.621+0000 7f28bb3bd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:19.620 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:19.623+0000 7f28bb3bd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:19.621 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:19.624+0000 7f28bb3bd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:19.621 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:19.624+0000 7f28bb3bd900 -1 bdev(0x560cf2952c00 td/test-erasure-code-plugins/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:19.621 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:19.624+0000 7f28bb3bd900 -1 bluestore(td/test-erasure-code-plugins/0) _read_fsid unparsable uuid 2026-02-19T13:10:20.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code-plugins/0/keyring 2026-02-19T13:10:20.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:20.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:20.045 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:10:20.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code-plugins/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:20.341 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:10:20.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:20.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=76f002e3-b263-4d2f-897f-bcc7a72fb665 --auth-supported=none --mon-host=127.0.0.1:17110 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code-plugins/0 --osd-journal=td/test-erasure-code-plugins/0/journal --chdir= --run-dir=td/test-erasure-code-plugins '--admin-socket=/tmp/ceph-asok.12230/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code-plugins/$name.log' '--pid-file=td/test-erasure-code-plugins/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --osd_erasure_code_plugins=shec_sse4 2026-02-19T13:10:20.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:20.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:20.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:20.355 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:20.358+0000 7f480a11d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:20.356 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:20.359+0000 7f480a11d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:20.357 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:20.360+0000 7f480a11d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:20.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:20.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:20.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:20.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:20.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:20.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:20.558 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:20.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:20.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:20.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:20.578 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:20.581+0000 7f480a11d900 -1 Falling back to public interface 2026-02-19T13:10:20.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:20.897 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:20.900+0000 7f480a11d900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:21.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:21.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:21.776 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:21.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:21.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:21.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:21.996 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/700481976,v1:127.0.0.1:6803/700481976] [v2:127.0.0.1:6804/700481976,v1:127.0.0.1:6805/700481976] exists,up 1d08ddc0-5865-4290-8813-c729cf0f74c1 2026-02-19T13:10:21.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:21.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:21.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:21.998 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: get_asok_path osd.0 2026-02-19T13:10:21.999 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:21.999 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:21.999 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:21.999 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:21.999 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:21.999 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.12230/ceph-osd.0.asok 2026-02-19T13:10:21.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: CEPH_ARGS= 2026-02-19T13:10:21.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:49: TEST_preload_warning: ceph --admin-daemon /tmp/ceph-asok.12230/ceph-osd.0.asok log flush 2026-02-19T13:10:22.057 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:22.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:50: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/mon.a.log 2026-02-19T13:10:22.065 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:19.149+0000 7f94e5730dc0 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T13:10:22.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:51: TEST_preload_warning: grep 'WARNING: osd_erasure_code_plugins contains plugin shec_sse4' td/test-erasure-code-plugins/osd.0.log 2026-02-19T13:10:22.066 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:20.582+0000 7f480a11d900 0 WARNING: osd_erasure_code_plugins contains plugin shec_sse4 that is now deprecated. Please modify the value for osd_erasure_code_plugins to use shec instead. 2026-02-19T13:10:22.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:52: TEST_preload_warning: teardown td/test-erasure-code-plugins 2026-02-19T13:10:22.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:22.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:22.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:22.069 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:22.069 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:22.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:22.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:22.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:22.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:22.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:22.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:22.174 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:22.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:22.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:22.175 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:22.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:22.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:22.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:22.176 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:22.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:22.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:22.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:22.181 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:22.181 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.181 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:22.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code-plugins.sh:54: TEST_preload_warning: return 0 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/test-erasure-code-plugins 0 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code-plugins 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code-plugins KILL 2026-02-19T13:10:22.183 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:22.184 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:22.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:22.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:22.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:22.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:22.184 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:22.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:22.185 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:22.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:22.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:22.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:22.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:22.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:22.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:22.188 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:22.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:22.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T13:10:22.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code-plugins 2026-02-19T13:10:22.191 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:22.191 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.191 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.12230 2026-02-19T13:10:22.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.12230 2026-02-19T13:10:22.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:22.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:22.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T13:10:22.193 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T13:10:22.193 DEBUG:teuthology.orchestra.run.trial111:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T13:10:22.222 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-code.sh... 2026-02-19T13:10:22.222 DEBUG:teuthology.orchestra.run.trial111:workunit test erasure-code/test-erasure-code.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh 2026-02-19T13:10:22.286 INFO:tasks.workunit.client.0.trial111.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T13:10:22.292 INFO:tasks.workunit.client.0.trial111.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T13:10:22.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T13:10:22.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T13:10:22.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T13:10:22.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T13:10:22.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T13:10:22.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T13:10:22.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T13:10:22.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T13:10:22.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/test-erasure-code 2026-02-19T13:10:22.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:22: run: local dir=td/test-erasure-code 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:23: run: shift 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: export CEPH_MON=127.0.0.1:7101 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:25: run: CEPH_MON=127.0.0.1:7101 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:26: run: export CEPH_ARGS 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: uuidgen 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:27: run: CEPH_ARGS+='--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none ' 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:30: run: setup td/test-erasure-code 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-code 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-code 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-19T13:10:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:10:22.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-19T13:10:22.295 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:10:22.295 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:10:22.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:10:22.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:10:22.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:10:22.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:10:22.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:10:22.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:10:22.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:10:22.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-19T13:10:22.299 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:10:22.299 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.299 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:22.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37389 2026-02-19T13:10:22.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:10:22.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:10:22.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-code 2026-02-19T13:10:22.302 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:10:22.302 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.302 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:22.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.37389 2026-02-19T13:10:22.303 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:10:22.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T13:10:22.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T13:10:22.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:10:22.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-code 1' TERM HUP INT 2026-02-19T13:10:22.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:31: run: run_mon td/test-erasure-code a 2026-02-19T13:10:22.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-code 2026-02-19T13:10:22.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:10:22.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:10:22.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:10:22.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-code/a 2026-02-19T13:10:22.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-code/a --run-dir=td/test-erasure-code 2026-02-19T13:10:22.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:10:22.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:22.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:22.323 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:22.323 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.323 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:22.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:22.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-code/a '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-code/log --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:10:22.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:10:22.348 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:10:22.348 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:22.349 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37389/ceph-mon.a.asok 2026-02-19T13:10:22.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:22.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37389/ceph-mon.a.asok config get fsid 2026-02-19T13:10:22.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:10:22.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:10:22.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:10:22.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:10:22.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37389/ceph-mon.a.asok 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:10:22.408 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.37389/ceph-mon.a.asok config get mon_host 2026-02-19T13:10:22.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:32: run: run_mgr td/test-erasure-code x 2026-02-19T13:10:22.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-code 2026-02-19T13:10:22.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:10:22.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:10:22.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:10:22.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-code/x 2026-02-19T13:10:22.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:10:22.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:10:22.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:22.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:22.576 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:22.576 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.576 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:22.577 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:22.577 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:22.578 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-code/x '--log-file=td/test-erasure-code/$name.log' '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --run-dir=td/test-erasure-code '--pid-file=td/test-erasure-code/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: get_asok_path mon.a 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37389/ceph-mon.a.asok 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: CEPH_ARGS= 2026-02-19T13:10:22.594 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:34: run: ceph --admin-daemon /tmp/ceph-asok.37389/ceph-mon.a.asok log flush 2026-02-19T13:10:22.647 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:22.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:35: run: grep 'load: jerasure.*lrc' td/test-erasure-code/mon.a.log 2026-02-19T13:10:22.655 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:22.343+0000 7ff3fe479dc0 0 load: jerasure load: lrc 2026-02-19T13:10:22.655 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: seq 0 10 2026-02-19T13:10:22.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:22.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 0 2026-02-19T13:10:22.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:22.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:22.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:10:22.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:22.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/0 2026-02-19T13:10:22.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:22.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:22.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:22.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:22.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/0' 2026-02-19T13:10:22.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/0/journal' 2026-02-19T13:10:22.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:22.664 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:22.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:22.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:22.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:22.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:22.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/0 2026-02-19T13:10:22.666 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:22.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=93976d8f-8bd3-4b16-95a0-25e4cbe25100 2026-02-19T13:10:22.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 93976d8f-8bd3-4b16-95a0-25e4cbe25100' 2026-02-19T13:10:22.667 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 93976d8f-8bd3-4b16-95a0-25e4cbe25100 2026-02-19T13:10:22.667 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:22.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC+C5dpACcQKBAAI/r56O9LdzLTS8Wsi/Q6zg== 2026-02-19T13:10:22.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC+C5dpACcQKBAAI/r56O9LdzLTS8Wsi/Q6zg=="}' 2026-02-19T13:10:22.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 93976d8f-8bd3-4b16-95a0-25e4cbe25100 -i td/test-erasure-code/0/new.json 2026-02-19T13:10:22.779 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:22.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/0/new.json 2026-02-19T13:10:22.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+C5dpACcQKBAAI/r56O9LdzLTS8Wsi/Q6zg== --osd-uuid 93976d8f-8bd3-4b16-95a0-25e4cbe25100 2026-02-19T13:10:22.805 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:22.808+0000 7f3b994dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:22.806 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:22.809+0000 7f3b994dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:22.807 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:22.810+0000 7f3b994dc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:22.807 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:22.811+0000 7f3b994dc900 -1 bdev(0x55cd27a4ac00 td/test-erasure-code/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:22.808 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:22.811+0000 7f3b994dc900 -1 bluestore(td/test-erasure-code/0) _read_fsid unparsable uuid 2026-02-19T13:10:23.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/0/keyring 2026-02-19T13:10:23.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:23.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:10:23.254 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:10:23.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:23.554 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:10:23.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:10:23.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/0 --osd-journal=td/test-erasure-code/0/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:23.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:23.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:23.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:23.567 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:23.570+0000 7f58f8ca6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:23.569 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:23.572+0000 7f58f8ca6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:23.569 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:23.573+0000 7f58f8ca6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:23.762 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:23.765+0000 7f58f8ca6900 -1 Falling back to public interface 2026-02-19T13:10:23.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:10:23.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:23.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:10:23.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:23.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:23.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:23.783 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:23.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:23.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:23.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:23.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:24.146 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:24.149+0000 7f58f8ca6900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:10:25.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:25.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:25.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:25.002 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:25.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:25.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:10:25.225 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 5 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/2072246718,v1:127.0.0.1:6803/2072246718] [v2:127.0.0.1:6804/2072246718,v1:127.0.0.1:6805/2072246718] exists,up 93976d8f-8bd3-4b16-95a0-25e4cbe25100 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 1 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:10:25.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/1 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/1' 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/1/journal' 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:25.231 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:25.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:25.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:25.232 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:25.232 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:25.232 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:25.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:25.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:25.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/1 2026-02-19T13:10:25.234 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:25.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9cc92143-4b08-4bed-af7a-53035401832a 2026-02-19T13:10:25.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 9cc92143-4b08-4bed-af7a-53035401832a' 2026-02-19T13:10:25.234 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 9cc92143-4b08-4bed-af7a-53035401832a 2026-02-19T13:10:25.234 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:25.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDBC5dpMd9zDhAAtZ6+JyXVe+divjzFlYpaCQ== 2026-02-19T13:10:25.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDBC5dpMd9zDhAAtZ6+JyXVe+divjzFlYpaCQ=="}' 2026-02-19T13:10:25.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9cc92143-4b08-4bed-af7a-53035401832a -i td/test-erasure-code/1/new.json 2026-02-19T13:10:25.458 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:25.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/1/new.json 2026-02-19T13:10:25.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDBC5dpMd9zDhAAtZ6+JyXVe+divjzFlYpaCQ== --osd-uuid 9cc92143-4b08-4bed-af7a-53035401832a 2026-02-19T13:10:25.485 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:25.488+0000 7f054ebd5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:25.486 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:25.489+0000 7f054ebd5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:25.487 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:25.490+0000 7f054ebd5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:25.488 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:25.491+0000 7f054ebd5900 -1 bdev(0x56365782cc00 td/test-erasure-code/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:25.488 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:25.491+0000 7f054ebd5900 -1 bluestore(td/test-erasure-code/1) _read_fsid unparsable uuid 2026-02-19T13:10:25.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/1/keyring 2026-02-19T13:10:25.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:25.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:10:25.905 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:10:25.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:26.202 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:10:26.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:10:26.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/1 --osd-journal=td/test-erasure-code/1/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:26.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:26.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:26.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:26.217 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:26.220+0000 7f44d6e1d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:26.218 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:26.221+0000 7f44d6e1d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:26.219 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:26.222+0000 7f44d6e1d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:26.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:10:26.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:26.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:10:26.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:26.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:26.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:26.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:26.422 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:26.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:26.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:26.425 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:26.428+0000 7f44d6e1d900 -1 Falling back to public interface 2026-02-19T13:10:26.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:26.817 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:26.820+0000 7f44d6e1d900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:10:27.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:27.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:27.643 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:27.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:27.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:27.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:10:27.864 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 9 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1518875909,v1:127.0.0.1:6811/1518875909] [v2:127.0.0.1:6812/1518875909,v1:127.0.0.1:6813/1518875909] exists,up 9cc92143-4b08-4bed-af7a-53035401832a 2026-02-19T13:10:27.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:27.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:27.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:27.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:27.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 2 2026-02-19T13:10:27.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:27.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:27.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:10:27.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:27.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/2 2026-02-19T13:10:27.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:27.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:27.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:27.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:27.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/2' 2026-02-19T13:10:27.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/2/journal' 2026-02-19T13:10:27.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:27.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:27.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:27.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:27.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:27.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:27.875 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:27.875 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:27.875 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:27.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:27.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:27.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:27.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:27.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:27.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:27.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:27.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:27.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:27.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:27.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:27.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:27.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:27.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/2 2026-02-19T13:10:27.878 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:27.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6b11956b-153e-43cc-a26f-b884181df63d 2026-02-19T13:10:27.879 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 6b11956b-153e-43cc-a26f-b884181df63d 2026-02-19T13:10:27.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 6b11956b-153e-43cc-a26f-b884181df63d' 2026-02-19T13:10:27.880 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:27.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDDC5dpU8qQNBAABLSFnhFQ3mJaBcQOiRJ/Lw== 2026-02-19T13:10:27.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDDC5dpU8qQNBAABLSFnhFQ3mJaBcQOiRJ/Lw=="}' 2026-02-19T13:10:27.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6b11956b-153e-43cc-a26f-b884181df63d -i td/test-erasure-code/2/new.json 2026-02-19T13:10:28.100 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:10:28.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/2/new.json 2026-02-19T13:10:28.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDDC5dpU8qQNBAABLSFnhFQ3mJaBcQOiRJ/Lw== --osd-uuid 6b11956b-153e-43cc-a26f-b884181df63d 2026-02-19T13:10:28.127 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:28.129+0000 7ff2e36cb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:28.128 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:28.130+0000 7ff2e36cb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:28.129 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:28.131+0000 7ff2e36cb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:28.129 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:28.131+0000 7ff2e36cb900 -1 bdev(0x5645b1724c00 td/test-erasure-code/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:28.130 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:28.131+0000 7ff2e36cb900 -1 bluestore(td/test-erasure-code/2) _read_fsid unparsable uuid 2026-02-19T13:10:28.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/2/keyring 2026-02-19T13:10:28.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:28.552 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:10:28.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:10:28.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:28.854 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:10:28.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:10:28.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/2 --osd-journal=td/test-erasure-code/2/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:28.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:28.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:28.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:28.868 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:28.870+0000 7f6c103db900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:28.869 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:28.871+0000 7f6c103db900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:28.870 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:28.872+0000 7f6c103db900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:29.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:10:29.105 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:29.108+0000 7f6c103db900 -1 Falling back to public interface 2026-02-19T13:10:29.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:29.404 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:29.406+0000 7f6c103db900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:10:30.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:30.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:30.300 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:30.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:30.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:30.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:10:30.521 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/13822781,v1:127.0.0.1:6819/13822781] [v2:127.0.0.1:6820/13822781,v1:127.0.0.1:6821/13822781] exists,up 6b11956b-153e-43cc-a26f-b884181df63d 2026-02-19T13:10:30.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:30.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:30.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:30.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:30.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 3 2026-02-19T13:10:30.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/3 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:30.525 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/3 2026-02-19T13:10:30.526 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:30.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d3b73c6b-f060-417f-9593-6c174bf1058c 2026-02-19T13:10:30.527 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 d3b73c6b-f060-417f-9593-6c174bf1058c 2026-02-19T13:10:30.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 d3b73c6b-f060-417f-9593-6c174bf1058c' 2026-02-19T13:10:30.527 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:30.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDGC5dp0E4ZIBAArZid6k4/fkcSbptv9rY7sw== 2026-02-19T13:10:30.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDGC5dp0E4ZIBAArZid6k4/fkcSbptv9rY7sw=="}' 2026-02-19T13:10:30.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d3b73c6b-f060-417f-9593-6c174bf1058c -i td/test-erasure-code/3/new.json 2026-02-19T13:10:30.758 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:10:30.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/3/new.json 2026-02-19T13:10:30.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDGC5dp0E4ZIBAArZid6k4/fkcSbptv9rY7sw== --osd-uuid d3b73c6b-f060-417f-9593-6c174bf1058c 2026-02-19T13:10:30.786 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:30.788+0000 7f8981c5a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:30.788 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:30.790+0000 7f8981c5a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:30.789 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:30.791+0000 7f8981c5a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:30.789 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:30.791+0000 7f8981c5a900 -1 bdev(0x5598243d8c00 td/test-erasure-code/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:30.789 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:30.791+0000 7f8981c5a900 -1 bluestore(td/test-erasure-code/3) _read_fsid unparsable uuid 2026-02-19T13:10:31.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/3/keyring 2026-02-19T13:10:31.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:31.179 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:10:31.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:10:31.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:31.478 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:10:31.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:10:31.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:31.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:31.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:31.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:31.493 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:31.495+0000 7f27fc712900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:31.494 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:31.496+0000 7f27fc712900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:31.495 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:31.497+0000 7f27fc712900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:31.657 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:31.658+0000 7f27fc712900 -1 Falling back to public interface 2026-02-19T13:10:31.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:10:31.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:31.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:10:31.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:31.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:31.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:31.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:31.703 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:31.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:31.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:10:31.924 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:31.956 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:31.957+0000 7f27fc712900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:10:32.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:32.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:32.926 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:10:33.148 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 17 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3672755171,v1:127.0.0.1:6827/3672755171] [v2:127.0.0.1:6828/3672755171,v1:127.0.0.1:6829/3672755171] exists,up d3b73c6b-f060-417f-9593-6c174bf1058c 2026-02-19T13:10:33.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:33.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:33.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 4 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/4 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/4' 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/4/journal' 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:33.154 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:33.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:33.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:33.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/4 2026-02-19T13:10:33.156 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:33.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7b0274aa-4793-4025-b089-d3d17fb22def 2026-02-19T13:10:33.156 INFO:tasks.workunit.client.0.trial111.stdout:add osd4 7b0274aa-4793-4025-b089-d3d17fb22def 2026-02-19T13:10:33.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 7b0274aa-4793-4025-b089-d3d17fb22def' 2026-02-19T13:10:33.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:33.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDJC5dp+GLcCRAArS/9iXiKKi3BPIxcFVZDBA== 2026-02-19T13:10:33.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDJC5dp+GLcCRAArS/9iXiKKi3BPIxcFVZDBA=="}' 2026-02-19T13:10:33.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7b0274aa-4793-4025-b089-d3d17fb22def -i td/test-erasure-code/4/new.json 2026-02-19T13:10:33.380 INFO:tasks.workunit.client.0.trial111.stdout:4 2026-02-19T13:10:33.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/4/new.json 2026-02-19T13:10:33.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDJC5dp+GLcCRAArS/9iXiKKi3BPIxcFVZDBA== --osd-uuid 7b0274aa-4793-4025-b089-d3d17fb22def 2026-02-19T13:10:33.407 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:33.409+0000 7fadf0d89900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:33.408 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:33.410+0000 7fadf0d89900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:33.411+0000 7fadf0d89900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:33.411+0000 7fadf0d89900 -1 bdev(0x55d09d8fac00 td/test-erasure-code/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:33.409 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:33.411+0000 7fadf0d89900 -1 bluestore(td/test-erasure-code/4) _read_fsid unparsable uuid 2026-02-19T13:10:33.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/4/keyring 2026-02-19T13:10:33.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:33.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T13:10:33.813 INFO:tasks.workunit.client.0.trial111.stdout:adding osd4 key to auth repository 2026-02-19T13:10:33.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:34.108 INFO:tasks.workunit.client.0.trial111.stdout:start osd.4 2026-02-19T13:10:34.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T13:10:34.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/4 --osd-journal=td/test-erasure-code/4/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:34.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:34.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:34.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:34.124 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:34.125+0000 7f17fc1f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:34.125 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:34.127+0000 7f17fc1f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:34.126 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:34.128+0000 7f17fc1f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:34.294 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:34.295+0000 7f17fc1f9900 -1 Falling back to public interface 2026-02-19T13:10:34.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T13:10:34.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:34.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T13:10:34.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:34.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:34.332 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:34.332 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:34.332 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:34.332 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:34.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:10:34.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:34.729 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:34.730+0000 7f17fc1f9900 -1 osd.4 0 log_to_monitors true 2026-02-19T13:10:35.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:35.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:35.551 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:35.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:35.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:35.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:10:35.772 INFO:tasks.workunit.client.0.trial111.stdout:osd.4 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1324287597,v1:127.0.0.1:6835/1324287597] [v2:127.0.0.1:6836/1324287597,v1:127.0.0.1:6837/1324287597] exists,up 7b0274aa-4793-4025-b089-d3d17fb22def 2026-02-19T13:10:35.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 5 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/5 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:35.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:35.777 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:35.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:35.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:35.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/5 2026-02-19T13:10:35.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:35.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6c3959d6-0e0a-41b2-b2eb-29761ed68995 2026-02-19T13:10:35.779 INFO:tasks.workunit.client.0.trial111.stdout:add osd5 6c3959d6-0e0a-41b2-b2eb-29761ed68995 2026-02-19T13:10:35.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 6c3959d6-0e0a-41b2-b2eb-29761ed68995' 2026-02-19T13:10:35.780 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:35.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDLC5dpd6ABLxAAzv2+SrK27eo8oaUAGvvXSA== 2026-02-19T13:10:35.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDLC5dpd6ABLxAAzv2+SrK27eo8oaUAGvvXSA=="}' 2026-02-19T13:10:35.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6c3959d6-0e0a-41b2-b2eb-29761ed68995 -i td/test-erasure-code/5/new.json 2026-02-19T13:10:36.007 INFO:tasks.workunit.client.0.trial111.stdout:5 2026-02-19T13:10:36.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/5/new.json 2026-02-19T13:10:36.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDLC5dpd6ABLxAAzv2+SrK27eo8oaUAGvvXSA== --osd-uuid 6c3959d6-0e0a-41b2-b2eb-29761ed68995 2026-02-19T13:10:36.034 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.035+0000 7f483fec1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:36.035 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.037+0000 7f483fec1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:36.036 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.038+0000 7f483fec1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:36.036 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.038+0000 7f483fec1900 -1 bdev(0x55a25d2fac00 td/test-erasure-code/5/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:36.036 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.038+0000 7f483fec1900 -1 bluestore(td/test-erasure-code/5) _read_fsid unparsable uuid 2026-02-19T13:10:36.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/5/keyring 2026-02-19T13:10:36.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T13:10:36.431 INFO:tasks.workunit.client.0.trial111.stdout:adding osd5 key to auth repository 2026-02-19T13:10:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:36.732 INFO:tasks.workunit.client.0.trial111.stdout:start osd.5 2026-02-19T13:10:36.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T13:10:36.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:36.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:36.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:36.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:36.746 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.748+0000 7f936d740900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:36.747 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.749+0000 7f936d740900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:36.748 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.750+0000 7f936d740900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:36.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T13:10:36.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:36.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T13:10:36.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:36.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:36.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:36.954 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:36.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:36.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:36.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:10:36.972 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:36.974+0000 7f936d740900 -1 Falling back to public interface 2026-02-19T13:10:37.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:37.371 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:37.373+0000 7f936d740900 -1 osd.5 0 log_to_monitors true 2026-02-19T13:10:38.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:38.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:38.177 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:38.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:38.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:38.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:10:38.401 INFO:tasks.workunit.client.0.trial111.stdout:osd.5 up in weight 1 up_from 25 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/4164417889,v1:127.0.0.1:6843/4164417889] [v2:127.0.0.1:6844/4164417889,v1:127.0.0.1:6845/4164417889] exists,up 6c3959d6-0e0a-41b2-b2eb-29761ed68995 2026-02-19T13:10:38.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:38.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:38.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:38.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:38.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 6 2026-02-19T13:10:38.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:38.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/6 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-19T13:10:38.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:38.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:38.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:38.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/6 2026-02-19T13:10:38.410 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:38.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=73cf7f6b-ea74-4ebc-9078-60f4379e69da 2026-02-19T13:10:38.410 INFO:tasks.workunit.client.0.trial111.stdout:add osd6 73cf7f6b-ea74-4ebc-9078-60f4379e69da 2026-02-19T13:10:38.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 73cf7f6b-ea74-4ebc-9078-60f4379e69da' 2026-02-19T13:10:38.411 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:38.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDOC5dpe1nzGBAAtqJvn/UBgX7Ei1huRj/yPg== 2026-02-19T13:10:38.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDOC5dpe1nzGBAAtqJvn/UBgX7Ei1huRj/yPg=="}' 2026-02-19T13:10:38.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 73cf7f6b-ea74-4ebc-9078-60f4379e69da -i td/test-erasure-code/6/new.json 2026-02-19T13:10:38.636 INFO:tasks.workunit.client.0.trial111.stdout:6 2026-02-19T13:10:38.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/6/new.json 2026-02-19T13:10:38.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDOC5dpe1nzGBAAtqJvn/UBgX7Ei1huRj/yPg== --osd-uuid 73cf7f6b-ea74-4ebc-9078-60f4379e69da 2026-02-19T13:10:38.663 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:38.665+0000 7fe8f28eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:38.664 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:38.666+0000 7fe8f28eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:38.665 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:38.667+0000 7fe8f28eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:38.666 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:38.667+0000 7fe8f28eb900 -1 bdev(0x55e7c2bcec00 td/test-erasure-code/6/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:38.666 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:38.667+0000 7fe8f28eb900 -1 bluestore(td/test-erasure-code/6) _read_fsid unparsable uuid 2026-02-19T13:10:39.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/6/keyring 2026-02-19T13:10:39.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:39.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T13:10:39.093 INFO:tasks.workunit.client.0.trial111.stdout:adding osd6 key to auth repository 2026-02-19T13:10:39.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:39.390 INFO:tasks.workunit.client.0.trial111.stdout:start osd.6 2026-02-19T13:10:39.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T13:10:39.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:39.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:39.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:39.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:39.405 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:39.407+0000 7ff697690900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:39.406 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:39.408+0000 7ff697690900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:39.407 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:39.409+0000 7ff697690900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:39.599 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:39.600+0000 7ff697690900 -1 Falling back to public interface 2026-02-19T13:10:39.612 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T13:10:39.612 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:39.612 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T13:10:39.612 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:39.612 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:39.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:39.613 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:39.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:39.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:39.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:10:39.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:39.975 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:39.977+0000 7ff697690900 -1 osd.6 0 log_to_monitors true 2026-02-19T13:10:40.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:40.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:40.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:40.834 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:40.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:40.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:10:41.056 INFO:tasks.workunit.client.0.trial111.stdout:osd.6 up in weight 1 up_from 29 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2326001816,v1:127.0.0.1:6851/2326001816] [v2:127.0.0.1:6852/2326001816,v1:127.0.0.1:6853/2326001816] exists,up 73cf7f6b-ea74-4ebc-9078-60f4379e69da 2026-02-19T13:10:41.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:41.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 7 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/7 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:41.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/7' 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/7/journal' 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:41.062 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:41.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:41.064 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/7 2026-02-19T13:10:41.064 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:41.064 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d29b41ee-5d51-4962-989a-47571cf6eedf 2026-02-19T13:10:41.064 INFO:tasks.workunit.client.0.trial111.stdout:add osd7 d29b41ee-5d51-4962-989a-47571cf6eedf 2026-02-19T13:10:41.064 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 d29b41ee-5d51-4962-989a-47571cf6eedf' 2026-02-19T13:10:41.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDRC5dpoB1gBBAAI1mO0qWM7FTzX4ZDGdIPiA== 2026-02-19T13:10:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDRC5dpoB1gBBAAI1mO0qWM7FTzX4ZDGdIPiA=="}' 2026-02-19T13:10:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d29b41ee-5d51-4962-989a-47571cf6eedf -i td/test-erasure-code/7/new.json 2026-02-19T13:10:41.288 INFO:tasks.workunit.client.0.trial111.stdout:7 2026-02-19T13:10:41.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/7/new.json 2026-02-19T13:10:41.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDRC5dpoB1gBBAAI1mO0qWM7FTzX4ZDGdIPiA== --osd-uuid d29b41ee-5d51-4962-989a-47571cf6eedf 2026-02-19T13:10:41.315 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:41.317+0000 7f5897bb9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:41.316 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:41.318+0000 7f5897bb9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:41.317 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:41.319+0000 7f5897bb9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:41.318 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:41.320+0000 7f5897bb9900 -1 bdev(0x55b6228e6c00 td/test-erasure-code/7/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:41.318 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:41.320+0000 7f5897bb9900 -1 bluestore(td/test-erasure-code/7) _read_fsid unparsable uuid 2026-02-19T13:10:41.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/7/keyring 2026-02-19T13:10:41.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:41.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-19T13:10:41.734 INFO:tasks.workunit.client.0.trial111.stdout:adding osd7 key to auth repository 2026-02-19T13:10:41.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:42.034 INFO:tasks.workunit.client.0.trial111.stdout:start osd.7 2026-02-19T13:10:42.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-19T13:10:42.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/7 --osd-journal=td/test-erasure-code/7/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:42.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:42.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:42.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:42.049 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:42.050+0000 7fb655b37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:42.051 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:42.053+0000 7fb655b37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:42.052 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:42.053+0000 7fb655b37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:42.253 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:42.254+0000 7fb655b37900 -1 Falling back to public interface 2026-02-19T13:10:42.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-19T13:10:42.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:42.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-19T13:10:42.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:42.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:42.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:42.255 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:42.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:42.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:42.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T13:10:42.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:42.620 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:42.622+0000 7fb655b37900 -1 osd.7 0 log_to_monitors true 2026-02-19T13:10:43.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:43.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:43.473 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:43.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:43.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:43.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T13:10:43.693 INFO:tasks.workunit.client.0.trial111.stdout:osd.7 up in weight 1 up_from 33 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/1824440735,v1:127.0.0.1:6859/1824440735] [v2:127.0.0.1:6860/1824440735,v1:127.0.0.1:6861/1824440735] exists,up d29b41ee-5d51-4962-989a-47571cf6eedf 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 8 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/8 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/8' 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/8/journal' 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:43.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:43.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/8 2026-02-19T13:10:43.696 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:43.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8060151f-cab1-4cc7-988e-7e44a113729c 2026-02-19T13:10:43.697 INFO:tasks.workunit.client.0.trial111.stdout:add osd8 8060151f-cab1-4cc7-988e-7e44a113729c 2026-02-19T13:10:43.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 8060151f-cab1-4cc7-988e-7e44a113729c' 2026-02-19T13:10:43.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:43.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDTC5dp/RFdKhAA52R3TgEdZADqiUVt/Qi9Eg== 2026-02-19T13:10:43.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDTC5dp/RFdKhAA52R3TgEdZADqiUVt/Qi9Eg=="}' 2026-02-19T13:10:43.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8060151f-cab1-4cc7-988e-7e44a113729c -i td/test-erasure-code/8/new.json 2026-02-19T13:10:43.927 INFO:tasks.workunit.client.0.trial111.stdout:8 2026-02-19T13:10:43.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/8/new.json 2026-02-19T13:10:43.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDTC5dp/RFdKhAA52R3TgEdZADqiUVt/Qi9Eg== --osd-uuid 8060151f-cab1-4cc7-988e-7e44a113729c 2026-02-19T13:10:43.953 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:43.955+0000 7fba427b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:43.954 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:43.956+0000 7fba427b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:43.955 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:43.957+0000 7fba427b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:43.956 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:43.957+0000 7fba427b6900 -1 bdev(0x5590a33e8c00 td/test-erasure-code/8/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:43.956 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:43.957+0000 7fba427b6900 -1 bluestore(td/test-erasure-code/8) _read_fsid unparsable uuid 2026-02-19T13:10:44.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/8/keyring 2026-02-19T13:10:44.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:44.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-19T13:10:44.360 INFO:tasks.workunit.client.0.trial111.stdout:adding osd8 key to auth repository 2026-02-19T13:10:44.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:44.658 INFO:tasks.workunit.client.0.trial111.stdout:start osd.8 2026-02-19T13:10:44.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-19T13:10:44.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/8 --osd-journal=td/test-erasure-code/8/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:44.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:44.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:44.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:44.673 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:44.674+0000 7f5a6d9e2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:44.674 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:44.675+0000 7f5a6d9e2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:44.674 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:44.676+0000 7f5a6d9e2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:44.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-19T13:10:44.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:44.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-19T13:10:44.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:44.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:44.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:44.882 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:44.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:44.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:44.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T13:10:44.890 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:44.891+0000 7f5a6d9e2900 -1 Falling back to public interface 2026-02-19T13:10:45.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:45.254 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:45.256+0000 7f5a6d9e2900 -1 osd.8 0 log_to_monitors true 2026-02-19T13:10:46.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:46.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:46.102 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:46.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:46.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:46.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T13:10:46.323 INFO:tasks.workunit.client.0.trial111.stdout:osd.8 up in weight 1 up_from 37 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/1389054879,v1:127.0.0.1:6867/1389054879] [v2:127.0.0.1:6868/1389054879,v1:127.0.0.1:6869/1389054879] exists,up 8060151f-cab1-4cc7-988e-7e44a113729c 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 9 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=9 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/9 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-19T13:10:46.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:46.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:46.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/9 2026-02-19T13:10:46.327 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:46.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=fdf33f8a-b54e-4729-a0a7-21291a902b92 2026-02-19T13:10:46.327 INFO:tasks.workunit.client.0.trial111.stdout:add osd9 fdf33f8a-b54e-4729-a0a7-21291a902b92 2026-02-19T13:10:46.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd9 fdf33f8a-b54e-4729-a0a7-21291a902b92' 2026-02-19T13:10:46.328 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:46.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDWC5dp7qlWFBAA26HE25DvnwxRDnvdU+cCZg== 2026-02-19T13:10:46.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDWC5dp7qlWFBAA26HE25DvnwxRDnvdU+cCZg=="}' 2026-02-19T13:10:46.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new fdf33f8a-b54e-4729-a0a7-21291a902b92 -i td/test-erasure-code/9/new.json 2026-02-19T13:10:46.558 INFO:tasks.workunit.client.0.trial111.stdout:9 2026-02-19T13:10:46.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/9/new.json 2026-02-19T13:10:46.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 9 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDWC5dp7qlWFBAA26HE25DvnwxRDnvdU+cCZg== --osd-uuid fdf33f8a-b54e-4729-a0a7-21291a902b92 2026-02-19T13:10:46.586 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:46.589+0000 7f6fd510e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:46.587 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:46.590+0000 7f6fd510e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:46.588 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:46.591+0000 7f6fd510e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:46.589 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:46.591+0000 7f6fd510e900 -1 bdev(0x55d369d50c00 td/test-erasure-code/9/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:46.589 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:46.592+0000 7f6fd510e900 -1 bluestore(td/test-erasure-code/9) _read_fsid unparsable uuid 2026-02-19T13:10:47.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/9/keyring 2026-02-19T13:10:47.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:47.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd9 key to auth repository 2026-02-19T13:10:47.006 INFO:tasks.workunit.client.0.trial111.stdout:adding osd9 key to auth repository 2026-02-19T13:10:47.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/9/keyring auth add osd.9 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:47.304 INFO:tasks.workunit.client.0.trial111.stdout:start osd.9 2026-02-19T13:10:47.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.9 2026-02-19T13:10:47.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 9 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:47.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:47.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:47.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:47.317 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:47.320+0000 7ff041cc8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:47.319 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:47.321+0000 7ff041cc8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:47.319 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:47.322+0000 7ff041cc8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 9 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:47.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T13:10:47.525 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:47.527+0000 7ff041cc8900 -1 Falling back to public interface 2026-02-19T13:10:47.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:47.912 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:47.915+0000 7ff041cc8900 -1 osd.9 0 log_to_monitors true 2026-02-19T13:10:48.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:48.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:48.746 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:48.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:48.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:48.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T13:10:48.976 INFO:tasks.workunit.client.0.trial111.stdout:osd.9 up in weight 1 up_from 41 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6874/607336226,v1:127.0.0.1:6875/607336226] [v2:127.0.0.1:6876/607336226,v1:127.0.0.1:6877/607336226] exists,up fdf33f8a-b54e-4729-a0a7-21291a902b92 2026-02-19T13:10:48.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:48.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:48.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:48.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:36: run: for id in $(seq 0 10) 2026-02-19T13:10:48.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:37: run: run_osd td/test-erasure-code 10 2026-02-19T13:10:48.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-code 2026-02-19T13:10:48.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:10:48.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=10 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-code/10 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-code/10' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-code/10/journal' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:10:48.979 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:10:48.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-code/10 2026-02-19T13:10:48.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:10:48.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=15a5ba11-202f-4718-8d13-d94d2adbdc3a 2026-02-19T13:10:48.981 INFO:tasks.workunit.client.0.trial111.stdout:add osd10 15a5ba11-202f-4718-8d13-d94d2adbdc3a 2026-02-19T13:10:48.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd10 15a5ba11-202f-4718-8d13-d94d2adbdc3a' 2026-02-19T13:10:48.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:10:48.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDYC5dpQEkrOxAA7bIKfu58jkzg4V3DR1U+gw== 2026-02-19T13:10:48.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDYC5dpQEkrOxAA7bIKfu58jkzg4V3DR1U+gw=="}' 2026-02-19T13:10:48.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 15a5ba11-202f-4718-8d13-d94d2adbdc3a -i td/test-erasure-code/10/new.json 2026-02-19T13:10:49.208 INFO:tasks.workunit.client.0.trial111.stdout:10 2026-02-19T13:10:49.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-code/10/new.json 2026-02-19T13:10:49.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 10 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDYC5dpQEkrOxAA7bIKfu58jkzg4V3DR1U+gw== --osd-uuid 15a5ba11-202f-4718-8d13-d94d2adbdc3a 2026-02-19T13:10:49.235 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:49.237+0000 7f163e7f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:49.236 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:49.239+0000 7f163e7f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:49.237 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:49.240+0000 7f163e7f9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:49.237 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:49.240+0000 7f163e7f9900 -1 bdev(0x559b66bc6c00 td/test-erasure-code/10/block) open stat got: (1) Operation not permitted 2026-02-19T13:10:49.238 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:49.240+0000 7f163e7f9900 -1 bluestore(td/test-erasure-code/10) _read_fsid unparsable uuid 2026-02-19T13:10:49.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-code/10/keyring 2026-02-19T13:10:49.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:10:49.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd10 key to auth repository 2026-02-19T13:10:49.621 INFO:tasks.workunit.client.0.trial111.stdout:adding osd10 key to auth repository 2026-02-19T13:10:49.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-code/10/keyring auth add osd.10 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:10:49.925 INFO:tasks.workunit.client.0.trial111.stdout:start osd.10 2026-02-19T13:10:49.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.10 2026-02-19T13:10:49.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 10 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/10 --osd-journal=td/test-erasure-code/10/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:10:49.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:10:49.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:10:49.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:10:49.940 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:49.942+0000 7f50836fa900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:49.941 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:49.943+0000 7f50836fa900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:49.942 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:49.944+0000 7f50836fa900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:10:50.136 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:50.137+0000 7f50836fa900 -1 Falling back to public interface 2026-02-19T13:10:50.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 10 2026-02-19T13:10:50.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:10:50.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=10 2026-02-19T13:10:50.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:10:50.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:10:50.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:50.147 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:10:50.148 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:10:50.148 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:50.148 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-19T13:10:50.370 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:10:50.520 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:10:50.521+0000 7f50836fa900 -1 osd.10 0 log_to_monitors true 2026-02-19T13:10:51.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:10:51.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:10:51.372 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:10:51.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:10:51.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:10:51.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.10 up' 2026-02-19T13:10:51.593 INFO:tasks.workunit.client.0.trial111.stdout:osd.10 up in weight 1 up_from 45 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6882/56003359,v1:127.0.0.1:6883/56003359] [v2:127.0.0.1:6884/56003359,v1:127.0.0.1:6885/56003359] exists,up 15a5ba11-202f-4718-8d13-d94d2adbdc3a 2026-02-19T13:10:51.594 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:10:51.594 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:10:51.595 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:10:51.595 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:39: run: create_rbd_pool 2026-02-19T13:10:51.595 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:526: create_rbd_pool: ceph osd pool delete rbd rbd --yes-i-really-really-mean-it 2026-02-19T13:10:51.804 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' does not exist 2026-02-19T13:10:51.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:527: create_rbd_pool: create_pool rbd 4 2026-02-19T13:10:51.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:10:52.036 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:10:52.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:10:53.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:528: create_rbd_pool: rbd pool init rbd 2026-02-19T13:10:53.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:40: run: wait_for_clean 2026-02-19T13:10:53.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:10:53.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:10:53.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:10:53.328 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:10:53.328 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:10:53.328 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:10:53.328 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:10:53.328 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:10:53.328 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:10:53.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:10:53.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:10:53.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:10:53.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:10:53.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:10:53.401 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:10:53.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:53.621 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:10:53.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836488 2026-02-19T13:10:53.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836488 2026-02-19T13:10:53.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488' 2026-02-19T13:10:53.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:53.703 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:10:53.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705672 2026-02-19T13:10:53.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705672 2026-02-19T13:10:53.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672' 2026-02-19T13:10:53.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:53.782 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:10:53.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-19T13:10:53.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-19T13:10:53.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855' 2026-02-19T13:10:53.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:53.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:10:53.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444039 2026-02-19T13:10:53.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444039 2026-02-19T13:10:53.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-73014444039' 2026-02-19T13:10:53.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:53.939 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:10:54.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313222 2026-02-19T13:10:54.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313222 2026-02-19T13:10:54.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-73014444039 4-90194313222' 2026-02-19T13:10:54.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:54.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:10:54.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182406 2026-02-19T13:10:54.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182406 2026-02-19T13:10:54.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-73014444039 4-90194313222 5-107374182406' 2026-02-19T13:10:54.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:54.096 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:10:54.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051589 2026-02-19T13:10:54.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051589 2026-02-19T13:10:54.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-73014444039 4-90194313222 5-107374182406 6-124554051589' 2026-02-19T13:10:54.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:54.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:10:54.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920774 2026-02-19T13:10:54.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920774 2026-02-19T13:10:54.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-73014444039 4-90194313222 5-107374182406 6-124554051589 7-141733920774' 2026-02-19T13:10:54.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:54.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:10:54.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789956 2026-02-19T13:10:54.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789956 2026-02-19T13:10:54.329 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-73014444039 4-90194313222 5-107374182406 6-124554051589 7-141733920774 8-158913789956' 2026-02-19T13:10:54.329 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:54.329 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:10:54.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659141 2026-02-19T13:10:54.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659141 2026-02-19T13:10:54.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-73014444039 4-90194313222 5-107374182406 6-124554051589 7-141733920774 8-158913789956 9-176093659141' 2026-02-19T13:10:54.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:10:54.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:10:54.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528323 2026-02-19T13:10:54.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528323 2026-02-19T13:10:54.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836488 1-38654705672 2-55834574855 3-73014444039 4-90194313222 5-107374182406 6-124554051589 7-141733920774 8-158913789956 9-176093659141 10-193273528323' 2026-02-19T13:10:54.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:54.485 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836488 2026-02-19T13:10:54.485 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:54.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:10:54.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836488 2026-02-19T13:10:54.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:54.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836488 2026-02-19T13:10:54.487 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 21474836488 2026-02-19T13:10:54.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836488' 2026-02-19T13:10:54.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:54.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836486 -lt 21474836488 2026-02-19T13:10:54.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:10:55.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:10:55.711 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:10:55.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836488 -lt 21474836488 2026-02-19T13:10:55.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:55.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705672 2026-02-19T13:10:55.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:55.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:10:55.935 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705672 2026-02-19T13:10:55.935 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:55.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705672 2026-02-19T13:10:55.936 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 38654705672 2026-02-19T13:10:55.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705672' 2026-02-19T13:10:55.937 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:10:56.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705672 -lt 38654705672 2026-02-19T13:10:56.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:56.156 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574855 2026-02-19T13:10:56.156 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:56.157 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:10:56.158 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574855 2026-02-19T13:10:56.158 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:56.159 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574855 2026-02-19T13:10:56.159 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 55834574855 2026-02-19T13:10:56.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574855' 2026-02-19T13:10:56.160 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:10:56.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-19T13:10:56.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:56.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-73014444039 2026-02-19T13:10:56.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:56.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:10:56.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-73014444039 2026-02-19T13:10:56.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:56.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444039 2026-02-19T13:10:56.386 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 73014444039 2026-02-19T13:10:56.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 73014444039' 2026-02-19T13:10:56.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:10:56.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444039 -lt 73014444039 2026-02-19T13:10:56.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:56.606 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313222 2026-02-19T13:10:56.606 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:56.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:10:56.608 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313222 2026-02-19T13:10:56.608 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:56.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313222 2026-02-19T13:10:56.609 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 90194313222 2026-02-19T13:10:56.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313222' 2026-02-19T13:10:56.610 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:10:56.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313222 -lt 90194313222 2026-02-19T13:10:56.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:56.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-107374182406 2026-02-19T13:10:56.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:56.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:10:56.835 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-107374182406 2026-02-19T13:10:56.835 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:56.836 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182406 2026-02-19T13:10:56.836 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 107374182406 2026-02-19T13:10:56.836 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 107374182406' 2026-02-19T13:10:56.836 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:10:57.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182406 -lt 107374182406 2026-02-19T13:10:57.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:57.059 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-124554051589 2026-02-19T13:10:57.059 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:57.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:10:57.060 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-124554051589 2026-02-19T13:10:57.060 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:57.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051589 2026-02-19T13:10:57.061 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 124554051589 2026-02-19T13:10:57.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 124554051589' 2026-02-19T13:10:57.062 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:10:57.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051590 -lt 124554051589 2026-02-19T13:10:57.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:57.284 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920774 2026-02-19T13:10:57.284 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:57.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:10:57.285 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920774 2026-02-19T13:10:57.285 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:57.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920774 2026-02-19T13:10:57.286 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.7 seq 141733920774 2026-02-19T13:10:57.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920774' 2026-02-19T13:10:57.287 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:10:57.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920774 -lt 141733920774 2026-02-19T13:10:57.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:57.513 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789956 2026-02-19T13:10:57.513 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:57.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:10:57.514 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789956 2026-02-19T13:10:57.515 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:57.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789956 2026-02-19T13:10:57.516 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.8 seq 158913789956 2026-02-19T13:10:57.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789956' 2026-02-19T13:10:57.516 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:10:57.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789957 -lt 158913789956 2026-02-19T13:10:57.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:57.737 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-176093659141 2026-02-19T13:10:57.737 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:57.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:10:57.739 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-176093659141 2026-02-19T13:10:57.739 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=176093659141 2026-02-19T13:10:57.740 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.9 seq 176093659141 2026-02-19T13:10:57.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 176093659141' 2026-02-19T13:10:57.740 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:10:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 176093659141 -lt 176093659141 2026-02-19T13:10:57.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:10:57.965 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528323 2026-02-19T13:10:57.965 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:10:57.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:10:57.967 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528323 2026-02-19T13:10:57.967 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:10:57.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528323 2026-02-19T13:10:57.968 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.10 seq 193273528323 2026-02-19T13:10:57.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528323' 2026-02-19T13:10:57.969 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:10:58.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528324 -lt 193273528323 2026-02-19T13:10:58.188 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:10:58.188 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:58.189 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:58.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 4 == 0 2026-02-19T13:10:58.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:10:58.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:10:58.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:10:58.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:10:58.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:10:58.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:10:58.487 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:10:58.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=4 2026-02-19T13:10:58.704 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:10:58.704 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:10:58.704 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:10:59.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 4 = 4 2026-02-19T13:10:59.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: get_asok_path osd.0 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.37389/ceph-osd.0.asok 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: CEPH_ARGS= 2026-02-19T13:10:59.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:42: run: ceph --admin-daemon /tmp/ceph-asok.37389/ceph-osd.0.asok log flush 2026-02-19T13:10:59.061 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:10:59.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:43: run: grep 'load: jerasure.*lrc' td/test-erasure-code/osd.0.log 2026-02-19T13:10:59.069 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:10:23.767+0000 7f58f8ca6900 0 load: jerasure load: lrc 2026-02-19T13:10:59.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:44: run: create_erasure_coded_pool ecpool 2026-02-19T13:10:59.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:56: create_erasure_coded_pool: local poolname=ecpool 2026-02-19T13:10:59.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:58: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile crush-failure-domain=osd 2026-02-19T13:10:59.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:60: create_erasure_coded_pool: create_pool ecpool 12 12 erasure myprofile 2026-02-19T13:10:59.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create ecpool 12 12 erasure myprofile 2026-02-19T13:10:59.681 INFO:tasks.workunit.client.0.trial111.stderr:pool 'ecpool' already exists 2026-02-19T13:10:59.693 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:11:00.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:62: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:11:00.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:00.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:00.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:00.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:00.696 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:00.696 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:00.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:00.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:00.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:00.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:00.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:00.989 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:00.989 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:11:00.989 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:11:00.989 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:11:00.989 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:11:00.989 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:11:00.990 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:11:00.990 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:11:00.990 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:11:00.990 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:11:00.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:00.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:00.990 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836492 2026-02-19T13:11:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836492 2026-02-19T13:11:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492' 2026-02-19T13:11:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.071 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:01.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705675 2026-02-19T13:11:01.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705675 2026-02-19T13:11:01.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675' 2026-02-19T13:11:01.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:01.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574859 2026-02-19T13:11:01.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574859 2026-02-19T13:11:01.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859' 2026-02-19T13:11:01.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:11:01.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444042 2026-02-19T13:11:01.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444042 2026-02-19T13:11:01.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-73014444042' 2026-02-19T13:11:01.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:11:01.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313226 2026-02-19T13:11:01.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313226 2026-02-19T13:11:01.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-73014444042 4-90194313226' 2026-02-19T13:11:01.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.388 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:11:01.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182409 2026-02-19T13:11:01.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182409 2026-02-19T13:11:01.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-73014444042 4-90194313226 5-107374182409' 2026-02-19T13:11:01.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.465 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:11:01.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051593 2026-02-19T13:11:01.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051593 2026-02-19T13:11:01.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-73014444042 4-90194313226 5-107374182409 6-124554051593' 2026-02-19T13:11:01.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:11:01.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920777 2026-02-19T13:11:01.626 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920777 2026-02-19T13:11:01.626 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-73014444042 4-90194313226 5-107374182409 6-124554051593 7-141733920777' 2026-02-19T13:11:01.626 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.626 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:11:01.707 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789960 2026-02-19T13:11:01.707 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789960 2026-02-19T13:11:01.707 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-73014444042 4-90194313226 5-107374182409 6-124554051593 7-141733920777 8-158913789960' 2026-02-19T13:11:01.707 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.707 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:11:01.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659144 2026-02-19T13:11:01.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659144 2026-02-19T13:11:01.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-73014444042 4-90194313226 5-107374182409 6-124554051593 7-141733920777 8-158913789960 9-176093659144' 2026-02-19T13:11:01.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:01.788 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:11:01.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528327 2026-02-19T13:11:01.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528327 2026-02-19T13:11:01.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836492 1-38654705675 2-55834574859 3-73014444042 4-90194313226 5-107374182409 6-124554051593 7-141733920777 8-158913789960 9-176093659144 10-193273528327' 2026-02-19T13:11:01.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:01.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836492 2026-02-19T13:11:01.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:01.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:01.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836492 2026-02-19T13:11:01.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:01.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836492 2026-02-19T13:11:01.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836492' 2026-02-19T13:11:01.869 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 21474836492 2026-02-19T13:11:01.869 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:02.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836490 -lt 21474836492 2026-02-19T13:11:02.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:03.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:03.095 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:03.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836492 -lt 21474836492 2026-02-19T13:11:03.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:03.316 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705675 2026-02-19T13:11:03.316 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:03.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:03.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705675 2026-02-19T13:11:03.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:03.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705675 2026-02-19T13:11:03.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705675' 2026-02-19T13:11:03.319 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 38654705675 2026-02-19T13:11:03.320 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:03.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705676 -lt 38654705675 2026-02-19T13:11:03.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:03.539 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574859 2026-02-19T13:11:03.539 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:03.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:03.540 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574859 2026-02-19T13:11:03.540 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:03.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574859 2026-02-19T13:11:03.541 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 55834574859 2026-02-19T13:11:03.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574859' 2026-02-19T13:11:03.542 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:03.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574859 -lt 55834574859 2026-02-19T13:11:03.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:03.764 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-73014444042 2026-02-19T13:11:03.764 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:03.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:11:03.765 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-73014444042 2026-02-19T13:11:03.765 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:03.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444042 2026-02-19T13:11:03.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 73014444042' 2026-02-19T13:11:03.767 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 73014444042 2026-02-19T13:11:03.767 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:11:03.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444042 -lt 73014444042 2026-02-19T13:11:03.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:03.991 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313226 2026-02-19T13:11:03.991 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:03.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:11:03.992 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313226 2026-02-19T13:11:03.992 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:03.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313226 2026-02-19T13:11:03.994 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 90194313226 2026-02-19T13:11:03.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313226' 2026-02-19T13:11:03.994 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:11:04.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313226 -lt 90194313226 2026-02-19T13:11:04.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:04.213 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-107374182409 2026-02-19T13:11:04.213 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:04.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:11:04.215 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-107374182409 2026-02-19T13:11:04.215 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:04.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182409 2026-02-19T13:11:04.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 107374182409' 2026-02-19T13:11:04.216 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 107374182409 2026-02-19T13:11:04.216 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:11:04.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182409 -lt 107374182409 2026-02-19T13:11:04.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:04.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-124554051593 2026-02-19T13:11:04.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:04.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:11:04.440 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-124554051593 2026-02-19T13:11:04.440 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:04.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051593 2026-02-19T13:11:04.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 124554051593' 2026-02-19T13:11:04.441 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 124554051593 2026-02-19T13:11:04.442 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:11:04.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051593 -lt 124554051593 2026-02-19T13:11:04.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:04.664 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920777 2026-02-19T13:11:04.664 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:04.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:11:04.665 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920777 2026-02-19T13:11:04.665 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:04.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920777 2026-02-19T13:11:04.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920777' 2026-02-19T13:11:04.667 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.7 seq 141733920777 2026-02-19T13:11:04.667 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:11:04.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920777 -lt 141733920777 2026-02-19T13:11:04.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:04.890 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789960 2026-02-19T13:11:04.890 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:04.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:11:04.892 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789960 2026-02-19T13:11:04.892 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:04.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789960 2026-02-19T13:11:04.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789960' 2026-02-19T13:11:04.893 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.8 seq 158913789960 2026-02-19T13:11:04.894 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:11:05.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789960 -lt 158913789960 2026-02-19T13:11:05.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:05.120 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-176093659144 2026-02-19T13:11:05.121 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:05.121 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:11:05.122 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-176093659144 2026-02-19T13:11:05.122 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:05.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=176093659144 2026-02-19T13:11:05.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 176093659144' 2026-02-19T13:11:05.124 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.9 seq 176093659144 2026-02-19T13:11:05.124 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:11:05.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 176093659145 -lt 176093659144 2026-02-19T13:11:05.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:05.347 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528327 2026-02-19T13:11:05.348 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:05.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:11:05.349 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528327 2026-02-19T13:11:05.349 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:05.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528327 2026-02-19T13:11:05.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528327' 2026-02-19T13:11:05.350 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.10 seq 193273528327 2026-02-19T13:11:05.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:11:05.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528327 -lt 193273528327 2026-02-19T13:11:05.570 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:05.571 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:05.571 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:05.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-19T13:11:05.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:05.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:05.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:05.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:05.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:05.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:05.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:06.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-19T13:11:06.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:06.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:06.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:06.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-19T13:11:06.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:06.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:06.388 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: set 2026-02-19T13:11:06.388 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T13:11:06.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:46: run: local 'funcs=TEST_alignment_constraints 2026-02-19T13:11:06.389 INFO:tasks.workunit.client.0.trial111.stderr:TEST_chunk_mapping 2026-02-19T13:11:06.389 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_put_get_isa 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_put_get_jerasure 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_put_get_lrc_advanced 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_put_get_lrc_kml 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_put_get_shec' 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_alignment_constraints td/test-erasure-code 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:253: TEST_alignment_constraints: local payload=ABC 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:254: TEST_alignment_constraints: echo ABC 2026-02-19T13:11:06.390 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:260: TEST_alignment_constraints: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T13:11:06.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:260: TEST_alignment_constraints: local stripe_unit=4096 2026-02-19T13:11:06.401 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: ceph osd erasure-code-profile get myprofile 2026-02-19T13:11:06.402 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: grep k= 2026-02-19T13:11:06.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: eval local k=2 2026-02-19T13:11:06.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:261: TEST_alignment_constraints: local k=2 2026-02-19T13:11:06.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:262: TEST_alignment_constraints: local block_size=8191 2026-02-19T13:11:06.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:263: TEST_alignment_constraints: dd if=/dev/zero of=td/test-erasure-code/ORIGINAL bs=8191 count=2 2026-02-19T13:11:06.629 INFO:tasks.workunit.client.0.trial111.stderr:2+0 records in 2026-02-19T13:11:06.629 INFO:tasks.workunit.client.0.trial111.stderr:2+0 records out 2026-02-19T13:11:06.629 INFO:tasks.workunit.client.0.trial111.stderr:16382 bytes (16 kB, 16 KiB) copied, 4.8248e-05 s, 340 MB/s 2026-02-19T13:11:06.630 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:264: TEST_alignment_constraints: rados --block-size=8191 --pool ecpool put UNALIGNED td/test-erasure-code/ORIGINAL 2026-02-19T13:11:06.645 INFO:tasks.workunit.client.0.trial111.stderr:INFO: op_size has been rounded to 8192 2026-02-19T13:11:06.649 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:266: TEST_alignment_constraints: rm td/test-erasure-code/ORIGINAL 2026-02-19T13:11:06.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T13:11:06.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_chunk_mapping td/test-erasure-code 2026-02-19T13:11:06.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:304: TEST_chunk_mapping: local dir=td/test-erasure-code 2026-02-19T13:11:06.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:311: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code ecpool 0 1 2026-02-19T13:11:06.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:281: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-19T13:11:06.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:282: verify_chunk_mapping: local poolname=ecpool 2026-02-19T13:11:06.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:283: verify_chunk_mapping: local first=0 2026-02-19T13:11:06.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:284: verify_chunk_mapping: local second=1 2026-02-19T13:11:06.651 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-19T13:11:06.652 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T13:11:06.661 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-19T13:11:06.661 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 FIRSTecpool 2026-02-19T13:11:06.661 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-19T13:11:06.662 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T13:11:06.671 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-19T13:11:06.672 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 SECONDecpool 2026-02-19T13:11:06.673 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: local 'payload= FIRSTecpool SECONDecpool' 2026-02-19T13:11:06.673 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:287: verify_chunk_mapping: echo -n ' FIRSTecpool SECONDecpool' 2026-02-19T13:11:06.673 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:289: verify_chunk_mapping: rados --pool ecpool put SOMETHINGecpool td/test-erasure-code/ORIGINAL 2026-02-19T13:11:06.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: rados --pool ecpool get SOMETHINGecpool td/test-erasure-code/COPY 2026-02-19T13:11:06.711 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-19T13:11:06.711 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:11:06.711 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-19T13:11:06.711 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-19T13:11:06.711 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:06.932 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:11:06.932 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:11:06.932 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:11:06.933 INFO:tasks.workunit.client.0.trial111.stderr:0' 2026-02-19T13:11:06.933 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-19T13:11:06.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-19T13:11:06.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local -a osds 2026-02-19T13:11:06.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i = 0 )) 2026-02-19T13:11:06.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-19T13:11:06.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.3 flush_journal 2026-02-19T13:11:06.998 INFO:tasks.workunit.client.0.trial111.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T13:11:07.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-19T13:11:07.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-19T13:11:07.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.5 flush_journal 2026-02-19T13:11:07.067 INFO:tasks.workunit.client.0.trial111.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T13:11:07.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-19T13:11:07.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-19T13:11:07.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-19T13:11:07.136 INFO:tasks.workunit.client.0.trial111.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T13:11:07.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-19T13:11:07.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-19T13:11:07.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.0 flush_journal 2026-02-19T13:11:07.206 INFO:tasks.workunit.client.0.trial111.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T13:11:07.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-19T13:11:07.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 4 )) 2026-02-19T13:11:07.208 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:11:07.208 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-19T13:11:07.209 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: get_osds ecpool SOMETHINGecpool 2026-02-19T13:11:07.210 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:11:07.210 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGecpool 2026-02-19T13:11:07.210 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHINGecpool 2026-02-19T13:11:07.210 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:07.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:11:07.431 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:11:07.431 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:11:07.431 INFO:tasks.workunit.client.0.trial111.stderr:0' 2026-02-19T13:11:07.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 9 0 2026-02-19T13:11:07.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: osds=('3' '5' '9' '0') 2026-02-19T13:11:07.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: local -a osds 2026-02-19T13:11:07.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: objectstore_tool td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: grep --quiet FIRSTecpool 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:11:07.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.3 2026-02-19T13:11:07.433 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:11:07.433 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:11:07.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:11:07.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:11:07.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:11:07.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:11:07.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 3 SOMETHINGecpool get-bytes 2026-02-19T13:11:07.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-19T13:11:07.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:07.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T13:11:07.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:07.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/3 2026-02-19T13:11:07.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/3 SOMETHINGecpool get-bytes 2026-02-19T13:11:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 3 2026-02-19T13:11:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-19T13:11:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:11:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T13:11:07.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/3 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/3' 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/3/journal' 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:11:07.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:11:07.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/3 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:start osd.3 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/3 --osd-journal=td/test-erasure-code/3/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/3/whoami 2026-02-19T13:11:07.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T13:11:07.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:11:07.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:11:07.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:11:07.859 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:07.861+0000 7f17ad9cc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:07.861 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:07.863+0000 7f17ad9cc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:07.862 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:07.864+0000 7f17ad9cc900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:08.074 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T13:11:08.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:11:08.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:11:08.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:11:08.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:11:08.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:08.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:11:08.075 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:11:08.076 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:08.076 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:11:08.076 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:08.078+0000 7f17ad9cc900 -1 Falling back to public interface 2026-02-19T13:11:08.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:08.490 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:08.492+0000 7f17ad9cc900 -1 osd.3 64 log_to_monitors true 2026-02-19T13:11:09.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:09.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:09.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:11:09.296 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:09.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:09.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:11:09.520 INFO:tasks.workunit.client.0.trial111.stderr:osd.3 up in weight 1 up_from 67 up_thru 67 down_at 64 last_clean_interval [17,64) [v2:127.0.0.1:6826/2200805400,v1:127.0.0.1:6827/2200805400] [v2:127.0.0.1:6828/2200805400,v1:127.0.0.1:6829/2200805400] exists,up d3b73c6b-f060-417f-9593-6c174bf1058c 2026-02-19T13:11:09.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:11:09.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:09.521 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:09.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:09.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:09.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:09.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:09.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:09.593 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:09.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:11:09.811 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:11:09.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:09.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:09.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:09.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836495 2026-02-19T13:11:09.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836495 2026-02-19T13:11:09.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495' 2026-02-19T13:11:09.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:09.891 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:09.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705679 2026-02-19T13:11:09.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705679 2026-02-19T13:11:09.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679' 2026-02-19T13:11:09.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:09.970 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:10.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574862 2026-02-19T13:11:10.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574862 2026-02-19T13:11:10.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862' 2026-02-19T13:11:10.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:10.047 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:11:10.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808835 2026-02-19T13:11:10.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808835 2026-02-19T13:11:10.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-287762808835' 2026-02-19T13:11:10.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:10.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:11:10.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313229 2026-02-19T13:11:10.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313229 2026-02-19T13:11:10.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-287762808835 4-90194313229' 2026-02-19T13:11:10.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:10.207 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:11:10.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182413 2026-02-19T13:11:10.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182413 2026-02-19T13:11:10.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-287762808835 4-90194313229 5-107374182413' 2026-02-19T13:11:10.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:10.286 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:11:10.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051596 2026-02-19T13:11:10.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051596 2026-02-19T13:11:10.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-287762808835 4-90194313229 5-107374182413 6-124554051596' 2026-02-19T13:11:10.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:10.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:11:10.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920781 2026-02-19T13:11:10.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920781 2026-02-19T13:11:10.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-287762808835 4-90194313229 5-107374182413 6-124554051596 7-141733920781' 2026-02-19T13:11:10.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:10.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:11:10.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789963 2026-02-19T13:11:10.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789963 2026-02-19T13:11:10.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-287762808835 4-90194313229 5-107374182413 6-124554051596 7-141733920781 8-158913789963' 2026-02-19T13:11:10.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:10.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:11:10.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659148 2026-02-19T13:11:10.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659148 2026-02-19T13:11:10.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-287762808835 4-90194313229 5-107374182413 6-124554051596 7-141733920781 8-158913789963 9-176093659148' 2026-02-19T13:11:10.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:10.601 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:11:10.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528330 2026-02-19T13:11:10.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528330 2026-02-19T13:11:10.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836495 1-38654705679 2-55834574862 3-287762808835 4-90194313229 5-107374182413 6-124554051596 7-141733920781 8-158913789963 9-176093659148 10-193273528330' 2026-02-19T13:11:10.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:10.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836495 2026-02-19T13:11:10.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:10.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:10.683 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836495 2026-02-19T13:11:10.683 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:10.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836495 2026-02-19T13:11:10.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836495' 2026-02-19T13:11:10.684 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 21474836495 2026-02-19T13:11:10.684 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:10.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836493 -lt 21474836495 2026-02-19T13:11:10.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:11.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:11.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:12.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836496 -lt 21474836495 2026-02-19T13:11:12.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:12.124 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705679 2026-02-19T13:11:12.124 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:12.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:12.125 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705679 2026-02-19T13:11:12.126 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:12.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705679 2026-02-19T13:11:12.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705679' 2026-02-19T13:11:12.127 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 38654705679 2026-02-19T13:11:12.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:12.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705679 -lt 38654705679 2026-02-19T13:11:12.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:12.348 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574862 2026-02-19T13:11:12.348 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:12.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:12.349 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574862 2026-02-19T13:11:12.349 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:12.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574862 2026-02-19T13:11:12.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574862' 2026-02-19T13:11:12.350 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 55834574862 2026-02-19T13:11:12.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:12.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574863 -lt 55834574862 2026-02-19T13:11:12.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:12.568 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808835 2026-02-19T13:11:12.569 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:12.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:11:12.570 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-287762808835 2026-02-19T13:11:12.570 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:12.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808835 2026-02-19T13:11:12.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 287762808835' 2026-02-19T13:11:12.571 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 287762808835 2026-02-19T13:11:12.571 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:11:12.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808835 -lt 287762808835 2026-02-19T13:11:12.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:12.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313229 2026-02-19T13:11:12.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:12.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:11:12.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313229 2026-02-19T13:11:12.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:12.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313229 2026-02-19T13:11:12.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313229' 2026-02-19T13:11:12.795 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.4 seq 90194313229 2026-02-19T13:11:12.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:11:13.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313230 -lt 90194313229 2026-02-19T13:11:13.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:13.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-107374182413 2026-02-19T13:11:13.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:13.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:11:13.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-107374182413 2026-02-19T13:11:13.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:13.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182413 2026-02-19T13:11:13.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 107374182413' 2026-02-19T13:11:13.019 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.5 seq 107374182413 2026-02-19T13:11:13.019 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:11:13.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182413 -lt 107374182413 2026-02-19T13:11:13.239 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:13.239 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-124554051596 2026-02-19T13:11:13.240 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:13.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:11:13.241 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-124554051596 2026-02-19T13:11:13.241 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:13.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051596 2026-02-19T13:11:13.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 124554051596' 2026-02-19T13:11:13.242 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.6 seq 124554051596 2026-02-19T13:11:13.242 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:11:13.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051597 -lt 124554051596 2026-02-19T13:11:13.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:13.460 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920781 2026-02-19T13:11:13.461 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:13.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:11:13.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920781 2026-02-19T13:11:13.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:13.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920781 2026-02-19T13:11:13.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920781' 2026-02-19T13:11:13.463 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.7 seq 141733920781 2026-02-19T13:11:13.463 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:11:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920781 -lt 141733920781 2026-02-19T13:11:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:13.684 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789963 2026-02-19T13:11:13.684 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:13.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:11:13.685 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789963 2026-02-19T13:11:13.685 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:13.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789963 2026-02-19T13:11:13.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789963' 2026-02-19T13:11:13.686 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.8 seq 158913789963 2026-02-19T13:11:13.686 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:11:13.906 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789964 -lt 158913789963 2026-02-19T13:11:13.906 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:13.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-176093659148 2026-02-19T13:11:13.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:13.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:11:13.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-176093659148 2026-02-19T13:11:13.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:13.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=176093659148 2026-02-19T13:11:13.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 176093659148' 2026-02-19T13:11:13.909 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.9 seq 176093659148 2026-02-19T13:11:13.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:11:14.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 176093659148 -lt 176093659148 2026-02-19T13:11:14.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:14.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528330 2026-02-19T13:11:14.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:14.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:11:14.133 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528330 2026-02-19T13:11:14.133 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:14.134 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528330 2026-02-19T13:11:14.134 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528330' 2026-02-19T13:11:14.134 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.10 seq 193273528330 2026-02-19T13:11:14.134 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:11:14.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528331 -lt 193273528330 2026-02-19T13:11:14.358 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:14.358 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:14.358 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:14.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-19T13:11:14.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:14.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:14.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:14.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:14.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:14.657 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:14.657 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:14.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-19T13:11:14.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:14.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:14.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:15.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-19T13:11:15.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:15.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:15.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: objectstore_tool td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-19T13:11:15.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-19T13:11:15.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:11:15.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=5 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: grep --quiet SECONDecpool 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=5 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.5 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:11:15.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:11:15.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:11:15.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:11:15.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 5 SOMETHINGecpool get-bytes 2026-02-19T13:11:15.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-19T13:11:15.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=5 2026-02-19T13:11:15.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:15.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/5 2026-02-19T13:11:15.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/5 SOMETHINGecpool get-bytes 2026-02-19T13:11:15.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 5 2026-02-19T13:11:15.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-19T13:11:15.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=5 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/5 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/5' 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/5/journal' 2026-02-19T13:11:15.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:11:15.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/5 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.5 2026-02-19T13:11:15.376 INFO:tasks.workunit.client.0.trial111.stderr:start osd.5 2026-02-19T13:11:15.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 5 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/5 --osd-journal=td/test-erasure-code/5/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:11:15.377 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/5/whoami 2026-02-19T13:11:15.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 5 = 5 ']' 2026-02-19T13:11:15.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:11:15.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:11:15.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:11:15.386 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:15.387+0000 7eff38f85900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:15.387 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:15.389+0000 7eff38f85900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:15.388 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:15.390+0000 7eff38f85900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:15.525 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:15.526+0000 7eff38f85900 -1 Falling back to public interface 2026-02-19T13:11:15.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 5 2026-02-19T13:11:15.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:11:15.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T13:11:15.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:11:15.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:11:15.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:15.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:11:15.599 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:11:15.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:15.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:11:15.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:15.910 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:15.911+0000 7eff38f85900 -1 osd.5 69 log_to_monitors true 2026-02-19T13:11:16.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:16.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:16.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:11:16.818 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:16.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:16.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:11:17.039 INFO:tasks.workunit.client.0.trial111.stderr:osd.5 up in weight 1 up_from 72 up_thru 72 down_at 69 last_clean_interval [25,69) [v2:127.0.0.1:6842/169872848,v1:127.0.0.1:6843/169872848] [v2:127.0.0.1:6844/169872848,v1:127.0.0.1:6845/169872848] exists,up 6c3959d6-0e0a-41b2-b2eb-29761ed68995 2026-02-19T13:11:17.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:11:17.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:17.040 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:17.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:17.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:17.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:17.113 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:17.113 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:17.113 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:17.332 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:11:17.333 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:11:17.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:17.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.334 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:17.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836499 2026-02-19T13:11:17.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836499 2026-02-19T13:11:17.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499' 2026-02-19T13:11:17.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.410 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:17.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705682 2026-02-19T13:11:17.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705682 2026-02-19T13:11:17.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682' 2026-02-19T13:11:17.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.489 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:17.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574866 2026-02-19T13:11:17.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574866 2026-02-19T13:11:17.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866' 2026-02-19T13:11:17.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.567 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:11:17.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808838 2026-02-19T13:11:17.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808838 2026-02-19T13:11:17.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-287762808838' 2026-02-19T13:11:17.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:11:17.722 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313233 2026-02-19T13:11:17.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313233 2026-02-19T13:11:17.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-287762808838 4-90194313233' 2026-02-19T13:11:17.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.723 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:11:17.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645315 2026-02-19T13:11:17.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645315 2026-02-19T13:11:17.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-287762808838 4-90194313233 5-309237645315' 2026-02-19T13:11:17.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.801 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:11:17.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=124554051600 2026-02-19T13:11:17.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 124554051600 2026-02-19T13:11:17.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-287762808838 4-90194313233 5-309237645315 6-124554051600' 2026-02-19T13:11:17.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.878 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:11:17.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920784 2026-02-19T13:11:17.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920784 2026-02-19T13:11:17.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-287762808838 4-90194313233 5-309237645315 6-124554051600 7-141733920784' 2026-02-19T13:11:17.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:17.957 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:11:18.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789967 2026-02-19T13:11:18.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789967 2026-02-19T13:11:18.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-287762808838 4-90194313233 5-309237645315 6-124554051600 7-141733920784 8-158913789967' 2026-02-19T13:11:18.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:18.036 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:11:18.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659151 2026-02-19T13:11:18.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659151 2026-02-19T13:11:18.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-287762808838 4-90194313233 5-309237645315 6-124554051600 7-141733920784 8-158913789967 9-176093659151' 2026-02-19T13:11:18.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:18.117 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:11:18.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528334 2026-02-19T13:11:18.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528334 2026-02-19T13:11:18.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836499 1-38654705682 2-55834574866 3-287762808838 4-90194313233 5-309237645315 6-124554051600 7-141733920784 8-158913789967 9-176093659151 10-193273528334' 2026-02-19T13:11:18.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:18.195 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836499 2026-02-19T13:11:18.195 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:18.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:18.196 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836499 2026-02-19T13:11:18.196 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:18.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836499 2026-02-19T13:11:18.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836499' 2026-02-19T13:11:18.197 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 21474836499 2026-02-19T13:11:18.197 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:18.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836497 -lt 21474836499 2026-02-19T13:11:18.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:19.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:19.417 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:19.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836499 -lt 21474836499 2026-02-19T13:11:19.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:19.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705682 2026-02-19T13:11:19.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:19.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:19.647 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705682 2026-02-19T13:11:19.647 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:19.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705682 2026-02-19T13:11:19.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705682' 2026-02-19T13:11:19.649 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 38654705682 2026-02-19T13:11:19.649 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:19.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705683 -lt 38654705682 2026-02-19T13:11:19.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:19.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574866 2026-02-19T13:11:19.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:19.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:19.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574866 2026-02-19T13:11:19.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:19.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574866 2026-02-19T13:11:19.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574866' 2026-02-19T13:11:19.874 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 55834574866 2026-02-19T13:11:19.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:20.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574866 -lt 55834574866 2026-02-19T13:11:20.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:20.098 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808838 2026-02-19T13:11:20.098 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:20.099 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:11:20.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-287762808838 2026-02-19T13:11:20.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:20.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808838 2026-02-19T13:11:20.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 287762808838' 2026-02-19T13:11:20.100 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 287762808838 2026-02-19T13:11:20.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:11:20.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808839 -lt 287762808838 2026-02-19T13:11:20.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:20.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313233 2026-02-19T13:11:20.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:20.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:11:20.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313233 2026-02-19T13:11:20.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:20.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313233 2026-02-19T13:11:20.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313233' 2026-02-19T13:11:20.326 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.4 seq 90194313233 2026-02-19T13:11:20.326 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:11:20.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313233 -lt 90194313233 2026-02-19T13:11:20.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:20.553 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645315 2026-02-19T13:11:20.553 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:20.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:11:20.554 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-309237645315 2026-02-19T13:11:20.554 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:20.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645315 2026-02-19T13:11:20.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 309237645315' 2026-02-19T13:11:20.556 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.5 seq 309237645315 2026-02-19T13:11:20.556 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:11:20.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645315 -lt 309237645315 2026-02-19T13:11:20.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:20.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-124554051600 2026-02-19T13:11:20.780 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:20.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:11:20.781 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-124554051600 2026-02-19T13:11:20.781 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:20.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=124554051600 2026-02-19T13:11:20.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 124554051600' 2026-02-19T13:11:20.782 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.6 seq 124554051600 2026-02-19T13:11:20.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:11:21.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 124554051600 -lt 124554051600 2026-02-19T13:11:21.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:21.002 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920784 2026-02-19T13:11:21.002 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:21.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:11:21.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920784 2026-02-19T13:11:21.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:21.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920784 2026-02-19T13:11:21.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920784' 2026-02-19T13:11:21.004 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.7 seq 141733920784 2026-02-19T13:11:21.005 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:11:21.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920785 -lt 141733920784 2026-02-19T13:11:21.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:21.225 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789967 2026-02-19T13:11:21.226 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:21.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:11:21.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789967 2026-02-19T13:11:21.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:21.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789967 2026-02-19T13:11:21.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789967' 2026-02-19T13:11:21.228 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.8 seq 158913789967 2026-02-19T13:11:21.228 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:11:21.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789967 -lt 158913789967 2026-02-19T13:11:21.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:21.447 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-176093659151 2026-02-19T13:11:21.447 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:21.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:11:21.448 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-176093659151 2026-02-19T13:11:21.448 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:21.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=176093659151 2026-02-19T13:11:21.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 176093659151' 2026-02-19T13:11:21.449 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.9 seq 176093659151 2026-02-19T13:11:21.450 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:11:21.672 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 176093659152 -lt 176093659151 2026-02-19T13:11:21.672 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:21.673 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528334 2026-02-19T13:11:21.673 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:21.673 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:11:21.674 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528334 2026-02-19T13:11:21.674 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:21.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528334 2026-02-19T13:11:21.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528334' 2026-02-19T13:11:21.675 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.10 seq 193273528334 2026-02-19T13:11:21.675 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:11:21.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528334 -lt 193273528334 2026-02-19T13:11:21.901 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:21.901 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:21.901 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:22.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 16 == 0 2026-02-19T13:11:22.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:22.198 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:22.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:22.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:22.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:22.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:22.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:22.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=16 2026-02-19T13:11:22.417 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:22.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:22.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:22.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 16 = 16 2026-02-19T13:11:22.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:22.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:22.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:313: TEST_chunk_mapping: ceph osd erasure-code-profile set remap-profile plugin=lrc 'layers=[ [ "cDD", "" ] ]' mapping=_DD 'crush-steps=[ [ "choose", "osd", 0 ] ]' 2026-02-19T13:11:23.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:318: TEST_chunk_mapping: ceph osd erasure-code-profile get remap-profile 2026-02-19T13:11:23.219 INFO:tasks.workunit.client.0.trial111.stdout:crush-device-class= 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:crush-failure-domain=host 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:crush-num-failure-domains=0 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:crush-osds-per-failure-domain=0 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:crush-root=default 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:crush-steps=[ [ "choose", "osd", 0 ] ] 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:k=-1 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:l=-1 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:layers=[ [ "cDD", "" ] ] 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:m=-1 2026-02-19T13:11:23.220 INFO:tasks.workunit.client.0.trial111.stdout:mapping=_DD 2026-02-19T13:11:23.221 INFO:tasks.workunit.client.0.trial111.stdout:plugin=lrc 2026-02-19T13:11:23.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:319: TEST_chunk_mapping: create_pool remap-pool 12 12 erasure remap-profile 2026-02-19T13:11:23.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create remap-pool 12 12 erasure remap-profile 2026-02-19T13:11:23.596 INFO:tasks.workunit.client.0.trial111.stderr:pool 'remap-pool' already exists 2026-02-19T13:11:23.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:11:24.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:327: TEST_chunk_mapping: verify_chunk_mapping td/test-erasure-code remap-pool 1 2 2026-02-19T13:11:24.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:281: verify_chunk_mapping: local dir=td/test-erasure-code 2026-02-19T13:11:24.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:282: verify_chunk_mapping: local poolname=remap-pool 2026-02-19T13:11:24.611 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:283: verify_chunk_mapping: local first=1 2026-02-19T13:11:24.611 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:284: verify_chunk_mapping: local second=2 2026-02-19T13:11:24.611 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-19T13:11:24.611 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T13:11:24.621 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-19T13:11:24.622 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 FIRSTremap-pool 2026-02-19T13:11:24.622 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: chunk_size 2026-02-19T13:11:24.622 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: ceph-conf --show-config-value osd_pool_erasure_code_stripe_unit 2026-02-19T13:11:24.632 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:270: chunk_size: echo 4096 2026-02-19T13:11:24.633 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: printf '%*s' 4096 SECONDremap-pool 2026-02-19T13:11:24.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:286: verify_chunk_mapping: local 'payload= FIRSTremap-pool SECONDremap-pool' 2026-02-19T13:11:24.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:287: verify_chunk_mapping: echo -n ' FIRSTremap-pool SECONDremap-pool' 2026-02-19T13:11:24.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:289: verify_chunk_mapping: rados --pool remap-pool put SOMETHINGremap-pool td/test-erasure-code/ORIGINAL 2026-02-19T13:11:24.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:290: verify_chunk_mapping: rados --pool remap-pool get SOMETHINGremap-pool td/test-erasure-code/COPY 2026-02-19T13:11:24.670 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-19T13:11:24.670 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-19T13:11:24.670 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-19T13:11:24.670 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-19T13:11:24.670 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:24.891 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-19T13:11:24.891 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:11:24.892 INFO:tasks.workunit.client.0.trial111.stderr:9' 2026-02-19T13:11:24.892 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-19T13:11:24.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-19T13:11:24.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:291: verify_chunk_mapping: local -a osds 2026-02-19T13:11:24.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i = 0 )) 2026-02-19T13:11:24.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-19T13:11:24.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.4 flush_journal 2026-02-19T13:11:24.960 INFO:tasks.workunit.client.0.trial111.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T13:11:24.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-19T13:11:24.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-19T13:11:24.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.6 flush_journal 2026-02-19T13:11:25.029 INFO:tasks.workunit.client.0.trial111.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T13:11:25.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-19T13:11:25.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-19T13:11:25.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:293: verify_chunk_mapping: ceph daemon osd.9 flush_journal 2026-02-19T13:11:25.099 INFO:tasks.workunit.client.0.trial111.stderr:Can't get admin socket path: unable to get conf option admin_socket for osd: b"error parsing 'osd': expected string of the form TYPE.ID, valid types are: auth, mon, osd, mds, mgr, client\n" 2026-02-19T13:11:25.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i++ )) 2026-02-19T13:11:25.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:292: verify_chunk_mapping: (( i < 3 )) 2026-02-19T13:11:25.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:295: verify_chunk_mapping: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:11:25.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:296: verify_chunk_mapping: rm td/test-erasure-code/COPY 2026-02-19T13:11:25.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: get_osds remap-pool SOMETHINGremap-pool 2026-02-19T13:11:25.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=remap-pool 2026-02-19T13:11:25.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHINGremap-pool 2026-02-19T13:11:25.104 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map remap-pool SOMETHINGremap-pool 2026-02-19T13:11:25.105 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:25.326 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-19T13:11:25.326 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:11:25.326 INFO:tasks.workunit.client.0.trial111.stderr:9' 2026-02-19T13:11:25.326 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 6 9 2026-02-19T13:11:25.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: osds=('4' '6' '9') 2026-02-19T13:11:25.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:298: verify_chunk_mapping: local -a osds 2026-02-19T13:11:25.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: objectstore_tool td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=6 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:299: verify_chunk_mapping: grep --quiet FIRSTremap-pool 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=6 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.6 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:11:25.327 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:11:25.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:11:25.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:11:25.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:11:25.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:11:25.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 6 SOMETHINGremap-pool get-bytes 2026-02-19T13:11:25.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-19T13:11:25.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:25.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=6 2026-02-19T13:11:25.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:25.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/6 2026-02-19T13:11:25.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/6 SOMETHINGremap-pool get-bytes 2026-02-19T13:11:25.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 6 2026-02-19T13:11:25.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-19T13:11:25.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:11:25.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=6 2026-02-19T13:11:25.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:11:25.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/6 2026-02-19T13:11:25.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/6' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/6/journal' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:25.758 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:11:25.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:11:25.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:11:25.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/6 2026-02-19T13:11:25.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.6 2026-02-19T13:11:25.760 INFO:tasks.workunit.client.0.trial111.stderr:start osd.6 2026-02-19T13:11:25.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 6 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/6 --osd-journal=td/test-erasure-code/6/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:11:25.760 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/6/whoami 2026-02-19T13:11:25.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 6 = 6 ']' 2026-02-19T13:11:25.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:11:25.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:11:25.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:11:25.770 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:25.771+0000 7f1099dce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:25.772 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:25.773+0000 7f1099dce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:25.773 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:25.774+0000 7f1099dce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:25.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 6 2026-02-19T13:11:25.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:11:25.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T13:11:25.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:11:25.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:11:25.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:25.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:11:25.979 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:11:25.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:25.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:11:25.991 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:25.992+0000 7f1099dce900 -1 Falling back to public interface 2026-02-19T13:11:26.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:26.358 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:26.359+0000 7f1099dce900 -1 osd.6 80 log_to_monitors true 2026-02-19T13:11:27.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:27.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:27.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:11:27.198 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:27.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:27.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:11:27.422 INFO:tasks.workunit.client.0.trial111.stderr:osd.6 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [29,80) [v2:127.0.0.1:6850/2222408911,v1:127.0.0.1:6851/2222408911] [v2:127.0.0.1:6852/2222408911,v1:127.0.0.1:6853/2222408911] exists,up 73cf7f6b-ea74-4ebc-9078-60f4379e69da 2026-02-19T13:11:27.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:11:27.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:11:27.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:11:27.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:27.423 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:27.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:27.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:27.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:27.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:27.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:27.495 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:27.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:27.714 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:27.714 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:27.715 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:27.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836503 2026-02-19T13:11:27.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836503 2026-02-19T13:11:27.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503' 2026-02-19T13:11:27.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:27.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:27.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705686 2026-02-19T13:11:27.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705686 2026-02-19T13:11:27.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686' 2026-02-19T13:11:27.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:27.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:27.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574870 2026-02-19T13:11:27.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574870 2026-02-19T13:11:27.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870' 2026-02-19T13:11:27.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:27.951 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:11:28.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808842 2026-02-19T13:11:28.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808842 2026-02-19T13:11:28.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-287762808842' 2026-02-19T13:11:28.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:28.028 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:11:28.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313237 2026-02-19T13:11:28.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313237 2026-02-19T13:11:28.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-287762808842 4-90194313237' 2026-02-19T13:11:28.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:28.108 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:11:28.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645319 2026-02-19T13:11:28.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645319 2026-02-19T13:11:28.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-287762808842 4-90194313237 5-309237645319' 2026-02-19T13:11:28.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:28.187 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:11:28.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285571 2026-02-19T13:11:28.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285571 2026-02-19T13:11:28.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-287762808842 4-90194313237 5-309237645319 6-356482285571' 2026-02-19T13:11:28.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:28.266 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:11:28.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920788 2026-02-19T13:11:28.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920788 2026-02-19T13:11:28.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-287762808842 4-90194313237 5-309237645319 6-356482285571 7-141733920788' 2026-02-19T13:11:28.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:28.344 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:11:28.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789971 2026-02-19T13:11:28.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789971 2026-02-19T13:11:28.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-287762808842 4-90194313237 5-309237645319 6-356482285571 7-141733920788 8-158913789971' 2026-02-19T13:11:28.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:28.428 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:11:28.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=176093659155 2026-02-19T13:11:28.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 176093659155 2026-02-19T13:11:28.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-287762808842 4-90194313237 5-309237645319 6-356482285571 7-141733920788 8-158913789971 9-176093659155' 2026-02-19T13:11:28.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:28.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:11:28.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528338 2026-02-19T13:11:28.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528338 2026-02-19T13:11:28.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836503 1-38654705686 2-55834574870 3-287762808842 4-90194313237 5-309237645319 6-356482285571 7-141733920788 8-158913789971 9-176093659155 10-193273528338' 2026-02-19T13:11:28.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:28.584 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836503 2026-02-19T13:11:28.584 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:28.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:28.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836503 2026-02-19T13:11:28.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:28.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836503 2026-02-19T13:11:28.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836503' 2026-02-19T13:11:28.587 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 21474836503 2026-02-19T13:11:28.587 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:28.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836501 -lt 21474836503 2026-02-19T13:11:28.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:29.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:29.810 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:30.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836503 -lt 21474836503 2026-02-19T13:11:30.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:30.032 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705686 2026-02-19T13:11:30.032 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:30.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:30.033 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705686 2026-02-19T13:11:30.033 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:30.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705686 2026-02-19T13:11:30.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705686' 2026-02-19T13:11:30.034 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 38654705686 2026-02-19T13:11:30.034 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:30.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705687 -lt 38654705686 2026-02-19T13:11:30.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:30.258 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574870 2026-02-19T13:11:30.258 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:30.259 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:30.260 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574870 2026-02-19T13:11:30.260 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:30.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574870 2026-02-19T13:11:30.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574870' 2026-02-19T13:11:30.261 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 55834574870 2026-02-19T13:11:30.261 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:30.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574870 -lt 55834574870 2026-02-19T13:11:30.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:30.484 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808842 2026-02-19T13:11:30.485 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:30.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:11:30.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-287762808842 2026-02-19T13:11:30.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:30.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808842 2026-02-19T13:11:30.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 287762808842' 2026-02-19T13:11:30.488 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 287762808842 2026-02-19T13:11:30.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:11:30.707 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808843 -lt 287762808842 2026-02-19T13:11:30.707 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:30.708 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313237 2026-02-19T13:11:30.708 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:30.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:11:30.709 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313237 2026-02-19T13:11:30.709 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:30.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313237 2026-02-19T13:11:30.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313237' 2026-02-19T13:11:30.710 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.4 seq 90194313237 2026-02-19T13:11:30.710 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:11:30.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313237 -lt 90194313237 2026-02-19T13:11:30.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:30.931 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645319 2026-02-19T13:11:30.931 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:30.932 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:11:30.933 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-309237645319 2026-02-19T13:11:30.933 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:30.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645319 2026-02-19T13:11:30.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 309237645319' 2026-02-19T13:11:30.934 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.5 seq 309237645319 2026-02-19T13:11:30.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:11:31.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645320 -lt 309237645319 2026-02-19T13:11:31.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:31.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-356482285571 2026-02-19T13:11:31.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:31.157 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:11:31.158 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-356482285571 2026-02-19T13:11:31.158 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:31.159 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285571 2026-02-19T13:11:31.159 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 356482285571' 2026-02-19T13:11:31.160 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.6 seq 356482285571 2026-02-19T13:11:31.160 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:11:31.380 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-19T13:11:31.380 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:31.381 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920788 2026-02-19T13:11:31.381 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:31.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:11:31.382 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920788 2026-02-19T13:11:31.382 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:31.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920788 2026-02-19T13:11:31.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920788' 2026-02-19T13:11:31.383 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.7 seq 141733920788 2026-02-19T13:11:31.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:11:31.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920789 -lt 141733920788 2026-02-19T13:11:31.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:31.604 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789971 2026-02-19T13:11:31.605 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:31.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:11:31.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789971 2026-02-19T13:11:31.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:31.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789971 2026-02-19T13:11:31.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789971' 2026-02-19T13:11:31.607 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.8 seq 158913789971 2026-02-19T13:11:31.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:11:31.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789971 -lt 158913789971 2026-02-19T13:11:31.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:31.827 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-176093659155 2026-02-19T13:11:31.827 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:31.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:11:31.828 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-176093659155 2026-02-19T13:11:31.828 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:31.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=176093659155 2026-02-19T13:11:31.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 176093659155' 2026-02-19T13:11:31.829 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.9 seq 176093659155 2026-02-19T13:11:31.830 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:11:32.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 176093659156 -lt 176093659155 2026-02-19T13:11:32.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:32.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528338 2026-02-19T13:11:32.051 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:32.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:11:32.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528338 2026-02-19T13:11:32.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528338 2026-02-19T13:11:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528338' 2026-02-19T13:11:32.053 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.10 seq 193273528338 2026-02-19T13:11:32.053 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:11:32.271 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528338 -lt 193273528338 2026-02-19T13:11:32.271 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:32.272 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:32.272 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:32.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T13:11:32.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:32.569 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:32.569 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:32.569 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:32.570 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:32.570 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:32.570 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:32.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T13:11:32.788 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:32.788 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:32.788 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:33.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T13:11:33.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:33.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: objectstore_tool td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-code 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=9 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:300: verify_chunk_mapping: grep --quiet SECONDremap-pool 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-code 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=9 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-code TERM osd.9 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:11:33.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:11:33.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:11:33.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-code 9 SOMETHINGremap-pool get-bytes 2026-02-19T13:11:33.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-code 2026-02-19T13:11:33.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:11:33.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=9 2026-02-19T13:11:33.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:11:33.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-code/9 2026-02-19T13:11:33.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-code/9 SOMETHINGremap-pool get-bytes 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-code 9 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-code 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=9 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-code/9 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false' 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:11:33.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-code/9' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-code/9/journal' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-code' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:11:33.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-code/$name.log' 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-code/$name.pid' 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-code/9 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.9 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:start osd.9 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 9 --fsid=6addb2a9-2158-467e-8d0b-bc67480a1286 --auth-supported=none --mon-host=127.0.0.1:7101 --mon-osd-prime-pg-temp=false --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-code/9 --osd-journal=td/test-erasure-code/9/journal --chdir= --run-dir=td/test-erasure-code '--admin-socket=/tmp/ceph-asok.37389/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-code/$name.log' '--pid-file=td/test-erasure-code/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:11:33.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-code/9/whoami 2026-02-19T13:11:33.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 9 = 9 ']' 2026-02-19T13:11:33.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:11:33.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:11:33.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:11:33.321 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:33.323+0000 7f461dc29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:33.322 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:33.324+0000 7f461dc29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:33.323 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:33.325+0000 7f461dc29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:11:33.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 9 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=9 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:33.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T13:11:33.535 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:33.536+0000 7f461dc29900 -1 Falling back to public interface 2026-02-19T13:11:33.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:11:33.868 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:11:33.869+0000 7f461dc29900 -1 osd.9 85 log_to_monitors true 2026-02-19T13:11:34.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:11:34.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:11:34.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:11:34.761 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:34.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:11:34.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.9 up' 2026-02-19T13:11:34.981 INFO:tasks.workunit.client.0.trial111.stderr:osd.9 up in weight 1 up_from 89 up_thru 89 down_at 86 last_clean_interval [41,85) [v2:127.0.0.1:6874/3451635590,v1:127.0.0.1:6875/3451635590] [v2:127.0.0.1:6876/3451635590,v1:127.0.0.1:6877/3451635590] exists,up fdf33f8a-b54e-4729-a0a7-21291a902b92 2026-02-19T13:11:34.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:11:34.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:11:34.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:11:34.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:11:34.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:11:34.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:11:34.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:11:34.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:11:34.982 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:11:34.982 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:11:34.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:11:34.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:11:34.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:11:35.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:11:35.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:11:35.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:11:35.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:11:35.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:11:35.055 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:11:35.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:11:35.274 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:35.274 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:11:35.274 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:11:35.274 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:11:35.274 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:11:35.274 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:11:35.275 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:11:35.275 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:11:35.275 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:11:35.275 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:11:35.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:11:35.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.275 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:11:35.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836507 2026-02-19T13:11:35.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836507 2026-02-19T13:11:35.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507' 2026-02-19T13:11:35.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.353 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:11:35.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705690 2026-02-19T13:11:35.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705690 2026-02-19T13:11:35.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690' 2026-02-19T13:11:35.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.434 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:11:35.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574873 2026-02-19T13:11:35.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574873 2026-02-19T13:11:35.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873' 2026-02-19T13:11:35.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:11:35.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808846 2026-02-19T13:11:35.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808846 2026-02-19T13:11:35.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873 3-287762808846' 2026-02-19T13:11:35.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.592 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:11:35.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313240 2026-02-19T13:11:35.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313240 2026-02-19T13:11:35.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873 3-287762808846 4-90194313240' 2026-02-19T13:11:35.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.671 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:11:35.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645322 2026-02-19T13:11:35.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645322 2026-02-19T13:11:35.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873 3-287762808846 4-90194313240 5-309237645322' 2026-02-19T13:11:35.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:11:35.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285574 2026-02-19T13:11:35.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285574 2026-02-19T13:11:35.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873 3-287762808846 4-90194313240 5-309237645322 6-356482285574' 2026-02-19T13:11:35.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.830 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:11:35.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920792 2026-02-19T13:11:35.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920792 2026-02-19T13:11:35.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873 3-287762808846 4-90194313240 5-309237645322 6-356482285574 7-141733920792' 2026-02-19T13:11:35.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:11:35.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789974 2026-02-19T13:11:35.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789974 2026-02-19T13:11:35.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873 3-287762808846 4-90194313240 5-309237645322 6-356482285574 7-141733920792 8-158913789974' 2026-02-19T13:11:35.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:35.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:11:36.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089347 2026-02-19T13:11:36.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089347 2026-02-19T13:11:36.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873 3-287762808846 4-90194313240 5-309237645322 6-356482285574 7-141733920792 8-158913789974 9-382252089347' 2026-02-19T13:11:36.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:11:36.070 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:11:36.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528341 2026-02-19T13:11:36.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528341 2026-02-19T13:11:36.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836507 1-38654705690 2-55834574873 3-287762808846 4-90194313240 5-309237645322 6-356482285574 7-141733920792 8-158913789974 9-382252089347 10-193273528341' 2026-02-19T13:11:36.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:36.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836507 2026-02-19T13:11:36.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:36.148 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:11:36.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836507 2026-02-19T13:11:36.149 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:36.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836507 2026-02-19T13:11:36.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836507' 2026-02-19T13:11:36.150 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 21474836507 2026-02-19T13:11:36.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:36.368 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836504 -lt 21474836507 2026-02-19T13:11:36.368 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:11:37.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:11:37.370 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:11:37.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836507 -lt 21474836507 2026-02-19T13:11:37.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:37.589 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705690 2026-02-19T13:11:37.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:37.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:11:37.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705690 2026-02-19T13:11:37.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:37.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705690 2026-02-19T13:11:37.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705690' 2026-02-19T13:11:37.591 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 38654705690 2026-02-19T13:11:37.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:11:37.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705690 -lt 38654705690 2026-02-19T13:11:37.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:37.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574873 2026-02-19T13:11:37.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:37.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:11:37.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574873 2026-02-19T13:11:37.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:37.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574873 2026-02-19T13:11:37.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574873' 2026-02-19T13:11:37.816 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 55834574873 2026-02-19T13:11:37.816 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:11:38.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574874 -lt 55834574873 2026-02-19T13:11:38.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:38.037 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808846 2026-02-19T13:11:38.037 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:38.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:11:38.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-287762808846 2026-02-19T13:11:38.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:38.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808846 2026-02-19T13:11:38.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 287762808846' 2026-02-19T13:11:38.039 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 287762808846 2026-02-19T13:11:38.039 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:11:38.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808846 -lt 287762808846 2026-02-19T13:11:38.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:38.262 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313240 2026-02-19T13:11:38.262 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:38.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:11:38.263 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313240 2026-02-19T13:11:38.263 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:38.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313240 2026-02-19T13:11:38.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313240' 2026-02-19T13:11:38.265 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.4 seq 90194313240 2026-02-19T13:11:38.265 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:11:38.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313241 -lt 90194313240 2026-02-19T13:11:38.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:38.489 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645322 2026-02-19T13:11:38.490 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:38.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:11:38.491 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-309237645322 2026-02-19T13:11:38.491 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:38.492 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645322 2026-02-19T13:11:38.492 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 309237645322' 2026-02-19T13:11:38.492 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.5 seq 309237645322 2026-02-19T13:11:38.492 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:11:38.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645323 -lt 309237645322 2026-02-19T13:11:38.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:38.714 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-356482285574 2026-02-19T13:11:38.714 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:38.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:11:38.715 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-356482285574 2026-02-19T13:11:38.715 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:38.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285574 2026-02-19T13:11:38.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 356482285574' 2026-02-19T13:11:38.716 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.6 seq 356482285574 2026-02-19T13:11:38.716 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:11:38.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285575 -lt 356482285574 2026-02-19T13:11:38.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:38.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920792 2026-02-19T13:11:38.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:38.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:11:38.939 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920792 2026-02-19T13:11:38.940 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:38.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920792 2026-02-19T13:11:38.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920792' 2026-02-19T13:11:38.941 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.7 seq 141733920792 2026-02-19T13:11:38.941 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:11:39.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920793 -lt 141733920792 2026-02-19T13:11:39.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:39.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789974 2026-02-19T13:11:39.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:39.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:11:39.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789974 2026-02-19T13:11:39.164 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:39.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789974 2026-02-19T13:11:39.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789974' 2026-02-19T13:11:39.165 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.8 seq 158913789974 2026-02-19T13:11:39.165 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:11:39.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789975 -lt 158913789974 2026-02-19T13:11:39.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:39.388 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-382252089347 2026-02-19T13:11:39.388 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:39.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:11:39.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-382252089347 2026-02-19T13:11:39.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:39.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089347 2026-02-19T13:11:39.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 382252089347' 2026-02-19T13:11:39.391 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.9 seq 382252089347 2026-02-19T13:11:39.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:11:39.614 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089348 -lt 382252089347 2026-02-19T13:11:39.615 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:11:39.615 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528341 2026-02-19T13:11:39.615 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:11:39.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:11:39.616 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528341 2026-02-19T13:11:39.616 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:11:39.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528341 2026-02-19T13:11:39.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528341' 2026-02-19T13:11:39.618 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.10 seq 193273528341 2026-02-19T13:11:39.618 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:11:39.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528342 -lt 193273528341 2026-02-19T13:11:39.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:11:39.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:39.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:40.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T13:11:40.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:11:40.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:11:40.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:11:40.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:11:40.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:11:40.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:11:40.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:11:40.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T13:11:40.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:11:40.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:11:40.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:11:40.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T13:11:40.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:11:40.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:11:40.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:329: TEST_chunk_mapping: delete_pool remap-pool 2026-02-19T13:11:40.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=remap-pool 2026-02-19T13:11:40.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete remap-pool remap-pool --yes-i-really-really-mean-it 2026-02-19T13:11:40.927 INFO:tasks.workunit.client.0.trial111.stderr:pool 'remap-pool' does not exist 2026-02-19T13:11:40.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:330: TEST_chunk_mapping: ceph osd erasure-code-profile rm remap-profile 2026-02-19T13:11:41.234 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile remap-profile does not exist 2026-02-19T13:11:41.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T13:11:41.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_isa td/test-erasure-code 2026-02-19T13:11:41.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:193: TEST_rados_put_get_isa: erasure_code_plugin_exists isa 2026-02-19T13:11:41.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2104: erasure_code_plugin_exists: local plugin=isa 2026-02-19T13:11:41.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2105: erasure_code_plugin_exists: local status 2026-02-19T13:11:41.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2106: erasure_code_plugin_exists: local grepstr 2026-02-19T13:11:41.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2107: erasure_code_plugin_exists: local s 2026-02-19T13:11:41.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: case `uname` in 2026-02-19T13:11:41.248 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2108: erasure_code_plugin_exists: uname 2026-02-19T13:11:41.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2110: erasure_code_plugin_exists: grepstr='isa.*No such file' 2026-02-19T13:11:41.248 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: ceph osd erasure-code-profile set TESTPROFILE plugin=isa 2026-02-19T13:11:41.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2113: erasure_code_plugin_exists: s= 2026-02-19T13:11:41.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2114: erasure_code_plugin_exists: local status=0 2026-02-19T13:11:41.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2115: erasure_code_plugin_exists: '[' 0 -eq 0 ']' 2026-02-19T13:11:41.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2116: erasure_code_plugin_exists: ceph osd erasure-code-profile rm TESTPROFILE 2026-02-19T13:11:41.822 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile TESTPROFILE does not exist 2026-02-19T13:11:41.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2125: erasure_code_plugin_exists: return 0 2026-02-19T13:11:41.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:197: TEST_rados_put_get_isa: local dir=td/test-erasure-code 2026-02-19T13:11:41.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:198: TEST_rados_put_get_isa: local poolname=pool-isa 2026-02-19T13:11:41.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:200: TEST_rados_put_get_isa: ceph osd erasure-code-profile set profile-isa plugin=isa crush-failure-domain=osd 2026-02-19T13:11:42.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:203: TEST_rados_put_get_isa: create_pool pool-isa 1 1 erasure profile-isa 2026-02-19T13:11:42.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-isa 1 1 erasure profile-isa 2026-02-19T13:11:42.471 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-isa' already exists 2026-02-19T13:11:42.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:11:43.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:206: TEST_rados_put_get_isa: rados_put_get td/test-erasure-code pool-isa 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-isa 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T13:11:43.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:43.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T13:11:43.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-isa put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T13:11:43.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:11:43.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:11:43.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T13:11:43.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-isa SOMETHING 2026-02-19T13:11:43.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-19T13:11:43.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:11:43.524 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-19T13:11:43.524 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr:10 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr:7' 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 7 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('4' '10' '0' '7') 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-19T13:11:43.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-19T13:11:44.010 INFO:tasks.workunit.client.0.trial111.stderr:osd.7 is already out. 2026-02-19T13:11:44.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T13:11:49.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-isa SOMETHING 2026-02-19T13:11:49.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-isa 2026-02-19T13:11:49.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:11:49.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-19T13:11:49.024 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-isa SOMETHING 2026-02-19T13:11:49.024 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:49.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=4 2026-02-19T13:11:49.245 INFO:tasks.workunit.client.0.trial111.stderr:10 2026-02-19T13:11:49.245 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:11:49.245 INFO:tasks.workunit.client.0.trial111.stderr:1' 2026-02-19T13:11:49.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 4 10 0 1 2026-02-19T13:11:49.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-isa get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:11:49.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:11:49.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-19T13:11:49.534 INFO:tasks.workunit.client.0.trial111.stderr:osd.7 is already in. 2026-02-19T13:11:49.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T13:11:49.548 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:208: TEST_rados_put_get_isa: delete_pool pool-isa 2026-02-19T13:11:49.548 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-isa 2026-02-19T13:11:49.548 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-isa pool-isa --yes-i-really-really-mean-it 2026-02-19T13:11:49.833 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-isa' does not exist 2026-02-19T13:11:49.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T13:11:49.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_jerasure td/test-erasure-code 2026-02-19T13:11:49.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:212: TEST_rados_put_get_jerasure: local dir=td/test-erasure-code 2026-02-19T13:11:49.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:214: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code ecpool 2026-02-19T13:11:49.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T13:11:49.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=ecpool 2026-02-19T13:11:49.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T13:11:49.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:49.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T13:11:49.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:49.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T13:11:49.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:49.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T13:11:49.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:49.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T13:11:49.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool ecpool put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T13:11:49.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:11:49.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:11:49.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T13:11:49.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds ecpool SOMETHING 2026-02-19T13:11:49.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:11:49.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:11:49.885 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T13:11:49.885 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:50.108 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-19T13:11:50.109 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:11:50.109 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:11:50.109 INFO:tasks.workunit.client.0.trial111.stderr:9' 2026-02-19T13:11:50.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 9 2026-02-19T13:11:50.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('6' '0' '7' '9') 2026-02-19T13:11:50.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T13:11:50.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=3 2026-02-19T13:11:50.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 9 2026-02-19T13:11:50.353 INFO:tasks.workunit.client.0.trial111.stderr:osd.9 is already out. 2026-02-19T13:11:50.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T13:11:55.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds ecpool SOMETHING 2026-02-19T13:11:55.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=ecpool 2026-02-19T13:11:55.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<9\>' 2026-02-19T13:11:55.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:11:55.367 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map ecpool SOMETHING 2026-02-19T13:11:55.368 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:55.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=6 2026-02-19T13:11:55.590 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:11:55.590 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:11:55.590 INFO:tasks.workunit.client.0.trial111.stderr:5' 2026-02-19T13:11:55.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 6 0 7 5 2026-02-19T13:11:55.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool ecpool get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:11:55.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:11:55.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 9 2026-02-19T13:11:55.838 INFO:tasks.workunit.client.0.trial111.stderr:osd.9 is already in. 2026-02-19T13:11:55.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T13:11:55.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:216: TEST_rados_put_get_jerasure: local poolname=pool-jerasure 2026-02-19T13:11:55.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:217: TEST_rados_put_get_jerasure: local profile=profile-jerasure 2026-02-19T13:11:55.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:219: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile set profile-jerasure plugin=jerasure k=4 m=2 crush-failure-domain=osd 2026-02-19T13:11:56.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:223: TEST_rados_put_get_jerasure: create_pool pool-jerasure 12 12 erasure profile-jerasure 2026-02-19T13:11:56.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 12 12 erasure profile-jerasure 2026-02-19T13:11:56.518 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:11:56.532 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:11:57.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:226: TEST_rados_put_get_jerasure: rados_put_get td/test-erasure-code pool-jerasure 2026-02-19T13:11:57.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T13:11:57.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-jerasure 2026-02-19T13:11:57.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T13:11:57.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:57.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T13:11:57.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:57.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T13:11:57.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:57.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T13:11:57.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:11:57.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T13:11:57.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T13:11:57.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:11:57.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:11:57.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T13:11:57.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-19T13:11:57.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:11:57.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:11:57.577 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T13:11:57.577 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:11:57.809 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T13:11:57.809 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:11:57.809 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:11:57.809 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:11:57.809 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:11:57.809 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:11:57.810 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-19T13:11:57.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '1' '9' '4' '3' '10') 2026-02-19T13:11:57.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T13:11:57.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=5 2026-02-19T13:11:57.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 10 2026-02-19T13:11:58.078 INFO:tasks.workunit.client.0.trial111.stderr:osd.10 is already out. 2026-02-19T13:11:58.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T13:12:03.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-jerasure SOMETHING 2026-02-19T13:12:03.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:12:03.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:03.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<10\>' 2026-02-19T13:12:03.094 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T13:12:03.094 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:03.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T13:12:03.319 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:12:03.320 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:03.320 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:12:03.320 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:03.320 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:12:03.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 6 2026-02-19T13:12:03.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:03.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:03.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 10 2026-02-19T13:12:03.626 INFO:tasks.workunit.client.0.trial111.stderr:osd.10 is already in. 2026-02-19T13:12:03.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:227: TEST_rados_put_get_jerasure: rados_osds_out_in td/test-erasure-code pool-jerasure 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:104: rados_osds_out_in: local dir=td/test-erasure-code 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:105: rados_osds_out_in: local poolname=pool-jerasure 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:106: rados_osds_out_in: local objname=SOMETHING 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 FFFF 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 GGGG 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 HHHH 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:109: rados_osds_out_in: for marker in FFFF GGGG HHHH IIII 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:110: rados_osds_out_in: printf '%*s' 1024 IIII 2026-02-19T13:12:03.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:116: rados_osds_out_in: rados --pool pool-jerasure put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T13:12:03.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:117: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:03.754 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:118: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:03.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:119: rados_osds_out_in: rm td/test-erasure-code/COPY 2026-02-19T13:12:03.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:127: rados_osds_out_in: wait_for_clean 2026-02-19T13:12:03.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:12:03.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:12:03.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:12:03.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:12:03.758 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:12:03.758 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:12:03.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:12:03.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:12:03.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:12:03.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:12:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:12:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:12:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:12:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:12:03.832 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:12:04.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:12:04.051 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:12:04.051 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:12:04.051 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:04.051 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:12:04.051 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:12:04.051 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:12:04.051 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:12:04.052 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:12:04.052 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:04.052 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:12:04.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:12:04.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:12:04.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836514 2026-02-19T13:12:04.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836514 2026-02-19T13:12:04.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514' 2026-02-19T13:12:04.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:12:04.209 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705698 2026-02-19T13:12:04.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705698 2026-02-19T13:12:04.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698' 2026-02-19T13:12:04.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.210 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:12:04.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574881 2026-02-19T13:12:04.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574881 2026-02-19T13:12:04.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881' 2026-02-19T13:12:04.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.289 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:12:04.370 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808854 2026-02-19T13:12:04.370 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808854 2026-02-19T13:12:04.370 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-287762808854' 2026-02-19T13:12:04.370 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.371 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:12:04.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313248 2026-02-19T13:12:04.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313248 2026-02-19T13:12:04.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-287762808854 4-90194313248' 2026-02-19T13:12:04.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.447 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:12:04.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645330 2026-02-19T13:12:04.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645330 2026-02-19T13:12:04.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-287762808854 4-90194313248 5-309237645330' 2026-02-19T13:12:04.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.527 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:12:04.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285582 2026-02-19T13:12:04.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285582 2026-02-19T13:12:04.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-287762808854 4-90194313248 5-309237645330 6-356482285582' 2026-02-19T13:12:04.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.604 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:12:04.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920800 2026-02-19T13:12:04.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920800 2026-02-19T13:12:04.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-287762808854 4-90194313248 5-309237645330 6-356482285582 7-141733920800' 2026-02-19T13:12:04.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:12:04.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789982 2026-02-19T13:12:04.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789982 2026-02-19T13:12:04.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-287762808854 4-90194313248 5-309237645330 6-356482285582 7-141733920800 8-158913789982' 2026-02-19T13:12:04.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.760 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:12:04.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089355 2026-02-19T13:12:04.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089355 2026-02-19T13:12:04.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-287762808854 4-90194313248 5-309237645330 6-356482285582 7-141733920800 8-158913789982 9-382252089355' 2026-02-19T13:12:04.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:04.844 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:12:04.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528349 2026-02-19T13:12:04.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528349 2026-02-19T13:12:04.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836514 1-38654705698 2-55834574881 3-287762808854 4-90194313248 5-309237645330 6-356482285582 7-141733920800 8-158913789982 9-382252089355 10-193273528349' 2026-02-19T13:12:04.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:04.919 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836514 2026-02-19T13:12:04.919 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:04.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:12:04.920 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836514 2026-02-19T13:12:04.920 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:04.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836514 2026-02-19T13:12:04.921 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 21474836514 2026-02-19T13:12:04.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836514' 2026-02-19T13:12:04.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:05.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836514 -lt 21474836514 2026-02-19T13:12:05.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:05.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705698 2026-02-19T13:12:05.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:05.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:12:05.149 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705698 2026-02-19T13:12:05.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:05.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705698 2026-02-19T13:12:05.151 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 38654705698 2026-02-19T13:12:05.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705698' 2026-02-19T13:12:05.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:12:05.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705698 -lt 38654705698 2026-02-19T13:12:05.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:05.373 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574881 2026-02-19T13:12:05.373 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:05.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:12:05.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574881 2026-02-19T13:12:05.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:05.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574881 2026-02-19T13:12:05.376 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 55834574881 2026-02-19T13:12:05.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574881' 2026-02-19T13:12:05.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:12:05.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574881 -lt 55834574881 2026-02-19T13:12:05.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:05.598 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808854 2026-02-19T13:12:05.598 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:05.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:12:05.600 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-287762808854 2026-02-19T13:12:05.600 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:05.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808854 2026-02-19T13:12:05.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 287762808854' 2026-02-19T13:12:05.601 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 287762808854 2026-02-19T13:12:05.601 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:12:05.824 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808854 -lt 287762808854 2026-02-19T13:12:05.824 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:05.825 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313248 2026-02-19T13:12:05.825 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:05.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:12:05.827 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313248 2026-02-19T13:12:05.827 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:05.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313248 2026-02-19T13:12:05.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313248' 2026-02-19T13:12:05.828 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 90194313248 2026-02-19T13:12:05.828 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:12:06.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313248 -lt 90194313248 2026-02-19T13:12:06.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:06.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645330 2026-02-19T13:12:06.051 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:06.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:12:06.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-309237645330 2026-02-19T13:12:06.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:06.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645330 2026-02-19T13:12:06.053 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 309237645330 2026-02-19T13:12:06.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 309237645330' 2026-02-19T13:12:06.054 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:12:06.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645330 -lt 309237645330 2026-02-19T13:12:06.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:06.275 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-356482285582 2026-02-19T13:12:06.275 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:06.276 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:12:06.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-356482285582 2026-02-19T13:12:06.277 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:06.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285582 2026-02-19T13:12:06.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 356482285582' 2026-02-19T13:12:06.278 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 356482285582 2026-02-19T13:12:06.279 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:12:06.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285582 -lt 356482285582 2026-02-19T13:12:06.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:06.499 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920800 2026-02-19T13:12:06.499 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:06.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:12:06.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920800 2026-02-19T13:12:06.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:06.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920800 2026-02-19T13:12:06.502 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.7 seq 141733920800 2026-02-19T13:12:06.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920800' 2026-02-19T13:12:06.503 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:12:06.722 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920800 -lt 141733920800 2026-02-19T13:12:06.722 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:06.723 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789982 2026-02-19T13:12:06.723 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:06.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:12:06.724 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789982 2026-02-19T13:12:06.724 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:06.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789982 2026-02-19T13:12:06.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789982' 2026-02-19T13:12:06.725 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.8 seq 158913789982 2026-02-19T13:12:06.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:12:06.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789982 -lt 158913789982 2026-02-19T13:12:06.949 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:06.949 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-382252089355 2026-02-19T13:12:06.949 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:06.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:12:06.951 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-382252089355 2026-02-19T13:12:06.951 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:06.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089355 2026-02-19T13:12:06.952 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.9 seq 382252089355 2026-02-19T13:12:06.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 382252089355' 2026-02-19T13:12:06.952 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:12:07.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089355 -lt 382252089355 2026-02-19T13:12:07.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:07.176 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528349 2026-02-19T13:12:07.176 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:07.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:12:07.177 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528349 2026-02-19T13:12:07.177 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:07.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528349 2026-02-19T13:12:07.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528349' 2026-02-19T13:12:07.178 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.10 seq 193273528349 2026-02-19T13:12:07.179 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:12:07.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528350 -lt 193273528349 2026-02-19T13:12:07.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:12:07.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:12:07.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:12:07.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T13:12:07.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:12:07.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:12:07.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:12:07.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:12:07.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:12:07.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:12:07.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:12:07.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T13:12:07.914 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:12:07.914 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:12:07.914 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:12:08.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T13:12:08.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:12:08.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:12:08.207 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-19T13:12:08.208 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:12:08.208 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:08.208 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T13:12:08.208 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:08.428 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T13:12:08.428 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:12:08.428 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:128: rados_osds_out_in: local 'osds_list=5 1 9 4 3 10' 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: osds=('5' '1' '9' '4' '3' '10') 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:129: rados_osds_out_in: local -a osds 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-19T13:12:08.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 5 2026-02-19T13:12:08.675 INFO:tasks.workunit.client.0.trial111.stderr:osd.5 is already out. 2026-02-19T13:12:08.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:130: rados_osds_out_in: for osd in 0 1 2026-02-19T13:12:08.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:131: rados_osds_out_in: ceph osd out 1 2026-02-19T13:12:08.980 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 is already out. 2026-02-19T13:12:08.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:133: rados_osds_out_in: wait_for_clean 2026-02-19T13:12:08.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:12:08.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:12:08.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:12:08.994 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:12:08.994 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:12:08.994 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:12:08.994 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:12:08.994 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:12:08.994 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:12:09.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:12:09.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:12:09.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:12:09.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:12:09.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:12:09.068 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:12:09.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:12:09.289 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:09.290 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:12:09.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:12:09.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.290 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:12:09.368 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836517 2026-02-19T13:12:09.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836517 2026-02-19T13:12:09.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517' 2026-02-19T13:12:09.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.369 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:12:09.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705701 2026-02-19T13:12:09.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705701 2026-02-19T13:12:09.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701' 2026-02-19T13:12:09.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.446 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:12:09.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574884 2026-02-19T13:12:09.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574884 2026-02-19T13:12:09.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884' 2026-02-19T13:12:09.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:12:09.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808857 2026-02-19T13:12:09.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808857 2026-02-19T13:12:09.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-287762808857' 2026-02-19T13:12:09.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.604 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:12:09.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313251 2026-02-19T13:12:09.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313251 2026-02-19T13:12:09.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-287762808857 4-90194313251' 2026-02-19T13:12:09.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.681 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:12:09.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645333 2026-02-19T13:12:09.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645333 2026-02-19T13:12:09.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-287762808857 4-90194313251 5-309237645333' 2026-02-19T13:12:09.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.761 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:12:09.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285585 2026-02-19T13:12:09.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285585 2026-02-19T13:12:09.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-287762808857 4-90194313251 5-309237645333 6-356482285585' 2026-02-19T13:12:09.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.841 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:12:09.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920803 2026-02-19T13:12:09.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920803 2026-02-19T13:12:09.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-287762808857 4-90194313251 5-309237645333 6-356482285585 7-141733920803' 2026-02-19T13:12:09.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:09.920 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:12:09.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789985 2026-02-19T13:12:10.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789985 2026-02-19T13:12:10.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-287762808857 4-90194313251 5-309237645333 6-356482285585 7-141733920803 8-158913789985' 2026-02-19T13:12:10.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:10.000 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:12:10.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089358 2026-02-19T13:12:10.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089358 2026-02-19T13:12:10.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-287762808857 4-90194313251 5-309237645333 6-356482285585 7-141733920803 8-158913789985 9-382252089358' 2026-02-19T13:12:10.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:10.079 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:12:10.159 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528352 2026-02-19T13:12:10.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528352 2026-02-19T13:12:10.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836517 1-38654705701 2-55834574884 3-287762808857 4-90194313251 5-309237645333 6-356482285585 7-141733920803 8-158913789985 9-382252089358 10-193273528352' 2026-02-19T13:12:10.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:10.160 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836517 2026-02-19T13:12:10.160 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:10.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:12:10.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836517 2026-02-19T13:12:10.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:10.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836517 2026-02-19T13:12:10.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836517' 2026-02-19T13:12:10.163 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 21474836517 2026-02-19T13:12:10.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:10.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836515 -lt 21474836517 2026-02-19T13:12:10.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:12:11.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:12:11.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:11.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836518 -lt 21474836517 2026-02-19T13:12:11.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:11.606 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705701 2026-02-19T13:12:11.606 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:11.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:12:11.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705701 2026-02-19T13:12:11.608 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:11.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705701 2026-02-19T13:12:11.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705701' 2026-02-19T13:12:11.609 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 38654705701 2026-02-19T13:12:11.609 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:12:11.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705701 -lt 38654705701 2026-02-19T13:12:11.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:11.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574884 2026-02-19T13:12:11.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:11.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:12:11.835 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574884 2026-02-19T13:12:11.835 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:11.836 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574884 2026-02-19T13:12:11.836 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574884' 2026-02-19T13:12:11.837 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 55834574884 2026-02-19T13:12:11.837 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:12:12.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574885 -lt 55834574884 2026-02-19T13:12:12.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:12.059 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808857 2026-02-19T13:12:12.060 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:12.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:12:12.061 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-287762808857 2026-02-19T13:12:12.061 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:12.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808857 2026-02-19T13:12:12.062 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 287762808857 2026-02-19T13:12:12.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 287762808857' 2026-02-19T13:12:12.063 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:12:12.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808857 -lt 287762808857 2026-02-19T13:12:12.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:12.284 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313251 2026-02-19T13:12:12.284 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:12.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:12:12.286 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313251 2026-02-19T13:12:12.286 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:12.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313251 2026-02-19T13:12:12.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313251' 2026-02-19T13:12:12.287 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 90194313251 2026-02-19T13:12:12.287 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:12:12.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313252 -lt 90194313251 2026-02-19T13:12:12.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:12.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645333 2026-02-19T13:12:12.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:12.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:12:12.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-309237645333 2026-02-19T13:12:12.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:12.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645333 2026-02-19T13:12:12.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 309237645333' 2026-02-19T13:12:12.511 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 309237645333 2026-02-19T13:12:12.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:12:12.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645334 -lt 309237645333 2026-02-19T13:12:12.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:12.733 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-356482285585 2026-02-19T13:12:12.733 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:12.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:12:12.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-356482285585 2026-02-19T13:12:12.735 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:12.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285585 2026-02-19T13:12:12.736 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 356482285585 2026-02-19T13:12:12.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 356482285585' 2026-02-19T13:12:12.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:12:12.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285585 -lt 356482285585 2026-02-19T13:12:12.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:12.959 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920803 2026-02-19T13:12:12.959 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:12.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:12:12.960 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920803 2026-02-19T13:12:12.960 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:12.961 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920803 2026-02-19T13:12:12.961 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920803' 2026-02-19T13:12:12.961 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.7 seq 141733920803 2026-02-19T13:12:12.962 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:12:13.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920803 -lt 141733920803 2026-02-19T13:12:13.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:13.182 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789985 2026-02-19T13:12:13.182 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:13.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:12:13.184 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789985 2026-02-19T13:12:13.184 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789985 2026-02-19T13:12:13.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789985' 2026-02-19T13:12:13.186 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.8 seq 158913789985 2026-02-19T13:12:13.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:12:13.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789986 -lt 158913789985 2026-02-19T13:12:13.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:13.417 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-382252089358 2026-02-19T13:12:13.417 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:13.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:12:13.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-382252089358 2026-02-19T13:12:13.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:13.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089358 2026-02-19T13:12:13.420 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.9 seq 382252089358 2026-02-19T13:12:13.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 382252089358' 2026-02-19T13:12:13.420 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:12:13.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089358 -lt 382252089358 2026-02-19T13:12:13.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:13.640 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528352 2026-02-19T13:12:13.640 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:13.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:12:13.641 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528352 2026-02-19T13:12:13.641 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:13.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528352 2026-02-19T13:12:13.642 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.10 seq 193273528352 2026-02-19T13:12:13.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528352' 2026-02-19T13:12:13.643 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:12:13.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528353 -lt 193273528352 2026-02-19T13:12:13.865 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:12:13.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:12:13.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:12:14.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T13:12:14.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:12:14.161 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:12:14.161 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:12:14.161 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:12:14.161 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:12:14.161 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:12:14.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:12:14.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T13:12:14.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:12:14.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:12:14.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:12:14.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T13:12:14.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:12:14.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:12:14.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-19T13:12:14.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-19T13:12:14.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:12:14.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:14.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<5\>' 2026-02-19T13:12:14.681 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T13:12:14.681 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:14.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-19T13:12:14.903 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:12:14.904 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:14.904 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:12:14.904 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:14.904 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:12:14.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-19T13:12:14.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:137: rados_osds_out_in: for osd in 0 1 2026-02-19T13:12:14.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-19T13:12:14.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:12:14.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:14.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:138: rados_osds_out_in: grep '\<1\>' 2026-02-19T13:12:14.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T13:12:14.906 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:15.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2 2026-02-19T13:12:15.127 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:12:15.127 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:15.127 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:12:15.127 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:15.127 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:12:15.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2 7 9 4 3 10 2026-02-19T13:12:15.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:140: rados_osds_out_in: rados --pool pool-jerasure get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:15.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:141: rados_osds_out_in: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:15.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-19T13:12:15.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 5 2026-02-19T13:12:15.419 INFO:tasks.workunit.client.0.trial111.stderr:osd.5 is already in. 2026-02-19T13:12:15.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:147: rados_osds_out_in: for osd in 0 1 2026-02-19T13:12:15.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:148: rados_osds_out_in: ceph osd in 1 2026-02-19T13:12:15.704 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 is already in. 2026-02-19T13:12:15.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:150: rados_osds_out_in: wait_for_clean 2026-02-19T13:12:15.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:12:15.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:12:15.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:12:15.717 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:12:15.718 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:12:15.718 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:12:15.718 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:12:15.718 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:12:15.718 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:12:15.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:12:15.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:12:15.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:12:15.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:12:15.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:12:15.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:12:16.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:8 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:16.012 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:12:16.013 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:12:16.013 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.013 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:12:16.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=21474836521 2026-02-19T13:12:16.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 21474836521 2026-02-19T13:12:16.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521' 2026-02-19T13:12:16.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:12:16.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=38654705704 2026-02-19T13:12:16.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 38654705704 2026-02-19T13:12:16.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704' 2026-02-19T13:12:16.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:12:16.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574888 2026-02-19T13:12:16.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574888 2026-02-19T13:12:16.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888' 2026-02-19T13:12:16.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.249 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:12:16.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=287762808860 2026-02-19T13:12:16.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 287762808860 2026-02-19T13:12:16.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-287762808860' 2026-02-19T13:12:16.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.327 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:12:16.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=90194313255 2026-02-19T13:12:16.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 90194313255 2026-02-19T13:12:16.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-287762808860 4-90194313255' 2026-02-19T13:12:16.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:12:16.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=309237645337 2026-02-19T13:12:16.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 309237645337 2026-02-19T13:12:16.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-287762808860 4-90194313255 5-309237645337' 2026-02-19T13:12:16.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:12:16.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285589 2026-02-19T13:12:16.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285589 2026-02-19T13:12:16.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-287762808860 4-90194313255 5-309237645337 6-356482285589' 2026-02-19T13:12:16.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.569 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:12:16.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=141733920806 2026-02-19T13:12:16.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 141733920806 2026-02-19T13:12:16.649 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-287762808860 4-90194313255 5-309237645337 6-356482285589 7-141733920806' 2026-02-19T13:12:16.649 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.649 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:12:16.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789989 2026-02-19T13:12:16.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789989 2026-02-19T13:12:16.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-287762808860 4-90194313255 5-309237645337 6-356482285589 7-141733920806 8-158913789989' 2026-02-19T13:12:16.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.728 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.9 flush_pg_stats 2026-02-19T13:12:16.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=382252089361 2026-02-19T13:12:16.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 382252089361 2026-02-19T13:12:16.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-287762808860 4-90194313255 5-309237645337 6-356482285589 7-141733920806 8-158913789989 9-382252089361' 2026-02-19T13:12:16.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:12:16.806 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.10 flush_pg_stats 2026-02-19T13:12:16.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=193273528356 2026-02-19T13:12:16.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 193273528356 2026-02-19T13:12:16.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-21474836521 1-38654705704 2-55834574888 3-287762808860 4-90194313255 5-309237645337 6-356482285589 7-141733920806 8-158913789989 9-382252089361 10-193273528356' 2026-02-19T13:12:16.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:16.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-21474836521 2026-02-19T13:12:16.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:16.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:12:16.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-21474836521 2026-02-19T13:12:16.885 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:16.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=21474836521 2026-02-19T13:12:16.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 21474836521' 2026-02-19T13:12:16.886 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 21474836521 2026-02-19T13:12:16.886 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:17.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836518 -lt 21474836521 2026-02-19T13:12:17.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:12:18.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:12:18.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:12:18.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 21474836521 -lt 21474836521 2026-02-19T13:12:18.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:18.331 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-38654705704 2026-02-19T13:12:18.331 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:18.332 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:12:18.333 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-38654705704 2026-02-19T13:12:18.333 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:18.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=38654705704 2026-02-19T13:12:18.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 38654705704' 2026-02-19T13:12:18.334 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 38654705704 2026-02-19T13:12:18.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:12:18.557 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 38654705704 -lt 38654705704 2026-02-19T13:12:18.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:18.559 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-55834574888 2026-02-19T13:12:18.559 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:18.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:12:18.560 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:18.560 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-55834574888 2026-02-19T13:12:18.561 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574888 2026-02-19T13:12:18.561 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 55834574888 2026-02-19T13:12:18.562 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 55834574888' 2026-02-19T13:12:18.562 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:12:18.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574888 -lt 55834574888 2026-02-19T13:12:18.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:18.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-287762808860 2026-02-19T13:12:18.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:18.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:12:18.785 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-287762808860 2026-02-19T13:12:18.785 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:18.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=287762808860 2026-02-19T13:12:18.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 287762808860' 2026-02-19T13:12:18.786 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 287762808860 2026-02-19T13:12:18.786 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:12:19.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 287762808860 -lt 287762808860 2026-02-19T13:12:19.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:19.006 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-90194313255 2026-02-19T13:12:19.006 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:19.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:12:19.008 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-90194313255 2026-02-19T13:12:19.008 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:19.009 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=90194313255 2026-02-19T13:12:19.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 90194313255' 2026-02-19T13:12:19.010 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 90194313255 2026-02-19T13:12:19.010 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:12:19.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 90194313255 -lt 90194313255 2026-02-19T13:12:19.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:19.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-309237645337 2026-02-19T13:12:19.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:19.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:12:19.234 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-309237645337 2026-02-19T13:12:19.234 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:19.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=309237645337 2026-02-19T13:12:19.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 309237645337' 2026-02-19T13:12:19.236 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 309237645337 2026-02-19T13:12:19.236 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:12:19.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 309237645337 -lt 309237645337 2026-02-19T13:12:19.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:19.455 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-356482285589 2026-02-19T13:12:19.456 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:19.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:12:19.457 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-356482285589 2026-02-19T13:12:19.457 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:19.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285589 2026-02-19T13:12:19.458 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 356482285589 2026-02-19T13:12:19.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 356482285589' 2026-02-19T13:12:19.459 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:12:19.676 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285589 -lt 356482285589 2026-02-19T13:12:19.676 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:19.677 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-141733920806 2026-02-19T13:12:19.677 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:19.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:12:19.678 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-141733920806 2026-02-19T13:12:19.678 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:19.679 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=141733920806 2026-02-19T13:12:19.679 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 141733920806' 2026-02-19T13:12:19.680 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.7 seq 141733920806 2026-02-19T13:12:19.680 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:12:19.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 141733920807 -lt 141733920806 2026-02-19T13:12:19.901 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:19.901 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-158913789989 2026-02-19T13:12:19.901 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:19.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:12:19.903 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-158913789989 2026-02-19T13:12:19.903 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:19.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789989 2026-02-19T13:12:19.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 158913789989' 2026-02-19T13:12:19.904 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.8 seq 158913789989 2026-02-19T13:12:19.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:12:20.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789989 -lt 158913789989 2026-02-19T13:12:20.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:20.125 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 9-382252089361 2026-02-19T13:12:20.125 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:20.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=9 2026-02-19T13:12:20.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 9-382252089361 2026-02-19T13:12:20.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:20.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=382252089361 2026-02-19T13:12:20.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.9 seq 382252089361' 2026-02-19T13:12:20.128 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.9 seq 382252089361 2026-02-19T13:12:20.128 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 9 2026-02-19T13:12:20.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 382252089362 -lt 382252089361 2026-02-19T13:12:20.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:12:20.353 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 10-193273528356 2026-02-19T13:12:20.353 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:12:20.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=10 2026-02-19T13:12:20.354 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 10-193273528356 2026-02-19T13:12:20.355 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:12:20.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=193273528356 2026-02-19T13:12:20.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.10 seq 193273528356' 2026-02-19T13:12:20.356 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.10 seq 193273528356 2026-02-19T13:12:20.357 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 10 2026-02-19T13:12:20.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 193273528356 -lt 193273528356 2026-02-19T13:12:20.583 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:12:20.583 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:12:20.583 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:12:20.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 28 == 0 2026-02-19T13:12:20.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:12:20.878 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:12:20.879 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:12:20.879 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:12:20.879 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:12:20.879 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:12:20.879 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:12:21.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=28 2026-02-19T13:12:21.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:12:21.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:12:21.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:12:21.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 28 = 28 2026-02-19T13:12:21.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:12:21.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:12:21.411 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: get_osds pool-jerasure SOMETHING 2026-02-19T13:12:21.411 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:12:21.412 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:21.412 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure SOMETHING 2026-02-19T13:12:21.412 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr:10' 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 1 9 4 3 10 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:151: rados_osds_out_in: test '5 1 9 4 3 10' = '5 1 9 4 3 10' 2026-02-19T13:12:21.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:152: rados_osds_out_in: rm td/test-erasure-code/ORIGINAL 2026-02-19T13:12:21.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:229: TEST_rados_put_get_jerasure: delete_pool pool-jerasure 2026-02-19T13:12:21.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-jerasure 2026-02-19T13:12:21.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:12:21.879 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:12:21.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:230: TEST_rados_put_get_jerasure: ceph osd erasure-code-profile rm profile-jerasure 2026-02-19T13:12:22.162 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile profile-jerasure does not exist 2026-02-19T13:12:22.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T13:12:22.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_advanced td/test-erasure-code 2026-02-19T13:12:22.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:156: TEST_rados_put_get_lrc_advanced: local dir=td/test-erasure-code 2026-02-19T13:12:22.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:157: TEST_rados_put_get_lrc_advanced: local poolname=pool-lrc-a 2026-02-19T13:12:22.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:158: TEST_rados_put_get_lrc_advanced: local profile=profile-lrc-a 2026-02-19T13:12:22.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:160: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile set profile-lrc-a plugin=lrc mapping=DD_ 'crush-steps=[ [ "chooseleaf", "osd", 0 ] ]' 'layers=[ [ "DDc", "" ] ]' 2026-02-19T13:12:22.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:165: TEST_rados_put_get_lrc_advanced: create_pool pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-19T13:12:22.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc-a 12 12 erasure profile-lrc-a 2026-02-19T13:12:22.823 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-lrc-a' already exists 2026-02-19T13:12:22.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:12:23.837 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:168: TEST_rados_put_get_lrc_advanced: rados_put_get td/test-erasure-code pool-lrc-a 2026-02-19T13:12:23.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T13:12:23.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc-a 2026-02-19T13:12:23.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T13:12:23.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:23.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T13:12:23.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:23.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T13:12:23.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:23.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T13:12:23.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:23.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T13:12:23.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc-a put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T13:12:23.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:23.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:23.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T13:12:23.877 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-19T13:12:23.877 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-19T13:12:23.877 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:23.877 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-19T13:12:23.878 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:24.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T13:12:24.102 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:24.102 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:12:24.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 3 2026-02-19T13:12:24.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('5' '9' '3') 2026-02-19T13:12:24.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T13:12:24.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-19T13:12:24.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 3 2026-02-19T13:12:24.348 INFO:tasks.workunit.client.0.trial111.stderr:osd.3 is already out. 2026-02-19T13:12:24.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T13:12:29.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc-a SOMETHING 2026-02-19T13:12:29.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc-a 2026-02-19T13:12:29.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:29.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<3\>' 2026-02-19T13:12:29.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc-a SOMETHING 2026-02-19T13:12:29.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:29.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=5 2026-02-19T13:12:29.586 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:29.586 INFO:tasks.workunit.client.0.trial111.stderr:7' 2026-02-19T13:12:29.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 5 9 7 2026-02-19T13:12:29.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc-a get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:29.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:29.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 3 2026-02-19T13:12:29.835 INFO:tasks.workunit.client.0.trial111.stderr:osd.3 is already in. 2026-02-19T13:12:29.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T13:12:29.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:170: TEST_rados_put_get_lrc_advanced: delete_pool pool-lrc-a 2026-02-19T13:12:29.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc-a 2026-02-19T13:12:29.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc-a pool-lrc-a --yes-i-really-really-mean-it 2026-02-19T13:12:30.132 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-lrc-a' does not exist 2026-02-19T13:12:30.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:171: TEST_rados_put_get_lrc_advanced: ceph osd erasure-code-profile rm profile-lrc-a 2026-02-19T13:12:30.412 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile profile-lrc-a does not exist 2026-02-19T13:12:30.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T13:12:30.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_lrc_kml td/test-erasure-code 2026-02-19T13:12:30.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:175: TEST_rados_put_get_lrc_kml: local dir=td/test-erasure-code 2026-02-19T13:12:30.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:176: TEST_rados_put_get_lrc_kml: local poolname=pool-lrc 2026-02-19T13:12:30.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:177: TEST_rados_put_get_lrc_kml: local profile=profile-lrc 2026-02-19T13:12:30.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:179: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile set profile-lrc plugin=lrc k=4 m=2 l=3 crush-failure-domain=osd 2026-02-19T13:12:30.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:183: TEST_rados_put_get_lrc_kml: create_pool pool-lrc 12 12 erasure profile-lrc 2026-02-19T13:12:30.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-lrc 12 12 erasure profile-lrc 2026-02-19T13:12:31.078 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-lrc' already exists 2026-02-19T13:12:31.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:12:32.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:186: TEST_rados_put_get_lrc_kml: rados_put_get td/test-erasure-code pool-lrc 2026-02-19T13:12:32.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T13:12:32.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-lrc 2026-02-19T13:12:32.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T13:12:32.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:32.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T13:12:32.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:32.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T13:12:32.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:32.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T13:12:32.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:32.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T13:12:32.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-lrc put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T13:12:32.114 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:32.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:32.133 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T13:12:32.134 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-19T13:12:32.134 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-19T13:12:32.135 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:32.135 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-19T13:12:32.136 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:32.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr:10 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr:7' 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 7 2026-02-19T13:12:32.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('8' '6' '0' '10' '5' '3' '9' '7') 2026-02-19T13:12:32.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T13:12:32.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=7 2026-02-19T13:12:32.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 7 2026-02-19T13:12:32.609 INFO:tasks.workunit.client.0.trial111.stderr:osd.7 is already out. 2026-02-19T13:12:32.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T13:12:37.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-lrc SOMETHING 2026-02-19T13:12:37.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-lrc 2026-02-19T13:12:37.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:37.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<7\>' 2026-02-19T13:12:37.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-lrc SOMETHING 2026-02-19T13:12:37.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:37.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=8 2026-02-19T13:12:37.849 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:12:37.849 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:12:37.849 INFO:tasks.workunit.client.0.trial111.stderr:10 2026-02-19T13:12:37.850 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:12:37.850 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:12:37.850 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:37.850 INFO:tasks.workunit.client.0.trial111.stderr:4' 2026-02-19T13:12:37.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 8 6 0 10 5 3 9 4 2026-02-19T13:12:37.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-lrc get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:37.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:37.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 7 2026-02-19T13:12:38.096 INFO:tasks.workunit.client.0.trial111.stderr:osd.7 is already in. 2026-02-19T13:12:38.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T13:12:38.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:188: TEST_rados_put_get_lrc_kml: delete_pool pool-lrc 2026-02-19T13:12:38.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-lrc 2026-02-19T13:12:38.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-lrc pool-lrc --yes-i-really-really-mean-it 2026-02-19T13:12:38.396 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-lrc' does not exist 2026-02-19T13:12:38.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:189: TEST_rados_put_get_lrc_kml: ceph osd erasure-code-profile rm profile-lrc 2026-02-19T13:12:38.676 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile profile-lrc does not exist 2026-02-19T13:12:38.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:47: run: for func in $funcs 2026-02-19T13:12:38.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:48: run: TEST_rados_put_get_shec td/test-erasure-code 2026-02-19T13:12:38.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:234: TEST_rados_put_get_shec: local dir=td/test-erasure-code 2026-02-19T13:12:38.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:236: TEST_rados_put_get_shec: local poolname=pool-shec 2026-02-19T13:12:38.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:237: TEST_rados_put_get_shec: local profile=profile-shec 2026-02-19T13:12:38.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:239: TEST_rados_put_get_shec: ceph osd erasure-code-profile set profile-shec plugin=shec k=2 m=1 c=1 crush-failure-domain=osd 2026-02-19T13:12:38.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:243: TEST_rados_put_get_shec: create_pool pool-shec 12 12 erasure profile-shec 2026-02-19T13:12:38.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-shec 12 12 erasure profile-shec 2026-02-19T13:12:39.363 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-shec' already exists 2026-02-19T13:12:39.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:12:40.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:246: TEST_rados_put_get_shec: rados_put_get td/test-erasure-code pool-shec 2026-02-19T13:12:40.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:66: rados_put_get: local dir=td/test-erasure-code 2026-02-19T13:12:40.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:67: rados_put_get: local poolname=pool-shec 2026-02-19T13:12:40.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:68: rados_put_get: local objname=SOMETHING 2026-02-19T13:12:40.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:40.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 AAA 2026-02-19T13:12:40.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:40.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 BBB 2026-02-19T13:12:40.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:40.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 CCCC 2026-02-19T13:12:40.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:71: rados_put_get: for marker in AAA BBB CCCC DDDD 2026-02-19T13:12:40.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:72: rados_put_get: printf '%*s' 1024 DDDD 2026-02-19T13:12:40.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:78: rados_put_get: rados --pool pool-shec put SOMETHING td/test-erasure-code/ORIGINAL 2026-02-19T13:12:40.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:79: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:40.415 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:80: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:40.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:81: rados_put_get: rm td/test-erasure-code/COPY 2026-02-19T13:12:40.417 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: get_osds pool-shec SOMETHING 2026-02-19T13:12:40.417 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-19T13:12:40.417 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:40.418 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-19T13:12:40.418 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:40.639 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-19T13:12:40.639 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:40.639 INFO:tasks.workunit.client.0.trial111.stderr:1' 2026-02-19T13:12:40.639 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 1 2026-02-19T13:12:40.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: initial_osds=('7' '9' '1') 2026-02-19T13:12:40.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:88: rados_put_get: local -a initial_osds 2026-02-19T13:12:40.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:89: rados_put_get: local last=2 2026-02-19T13:12:40.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:90: rados_put_get: ceph osd out 1 2026-02-19T13:12:40.885 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 is already out. 2026-02-19T13:12:40.897 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:93: rados_put_get: sleep 5 2026-02-19T13:12:45.899 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: get_osds pool-shec SOMETHING 2026-02-19T13:12:45.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-shec 2026-02-19T13:12:45.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=SOMETHING 2026-02-19T13:12:45.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:94: rados_put_get: grep '\<1\>' 2026-02-19T13:12:45.900 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-shec SOMETHING 2026-02-19T13:12:45.900 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:12:46.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=7 2026-02-19T13:12:46.124 INFO:tasks.workunit.client.0.trial111.stderr:9 2026-02-19T13:12:46.124 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:12:46.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 7 9 6 2026-02-19T13:12:46.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:96: rados_put_get: rados --pool pool-shec get SOMETHING td/test-erasure-code/COPY 2026-02-19T13:12:46.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:97: rados_put_get: diff td/test-erasure-code/ORIGINAL td/test-erasure-code/COPY 2026-02-19T13:12:46.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:98: rados_put_get: ceph osd in 1 2026-02-19T13:12:46.436 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 is already in. 2026-02-19T13:12:46.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:100: rados_put_get: rm td/test-erasure-code/ORIGINAL 2026-02-19T13:12:46.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:248: TEST_rados_put_get_shec: delete_pool pool-shec 2026-02-19T13:12:46.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=pool-shec 2026-02-19T13:12:46.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete pool-shec pool-shec --yes-i-really-really-mean-it 2026-02-19T13:12:46.713 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-shec' does not exist 2026-02-19T13:12:46.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:249: TEST_rados_put_get_shec: ceph osd erasure-code-profile rm profile-shec 2026-02-19T13:12:46.999 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile profile-shec does not exist 2026-02-19T13:12:47.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:51: run: delete_pool ecpool 2026-02-19T13:12:47.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:537: delete_pool: local poolname=ecpool 2026-02-19T13:12:47.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:538: delete_pool: ceph osd pool delete ecpool ecpool --yes-i-really-really-mean-it 2026-02-19T13:12:47.297 INFO:tasks.workunit.client.0.trial111.stderr:pool 'ecpool' does not exist 2026-02-19T13:12:47.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-code.sh:52: run: teardown td/test-erasure-code 2026-02-19T13:12:47.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-19T13:12:47.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:12:47.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-19T13:12:47.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:12:47.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:12:47.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:12:47.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:12:47.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:12:47.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:12:47.419 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:12:47.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:12:47.420 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:12:47.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:12:47.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:12:47.421 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:12:47.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:47.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:12:47.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:12:47.422 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:47.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:12:47.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:12:47.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-19T13:12:47.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:12:47.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:47.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:12:47.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37389 2026-02-19T13:12:47.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:12:47.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:12:47.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T13:12:47.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/test-erasure-code 0 2026-02-19T13:12:47.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-code 2026-02-19T13:12:47.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T13:12:47.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-code KILL 2026-02-19T13:12:47.482 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:12:47.482 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:12:47.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:12:47.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:12:47.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:12:47.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:12:47.483 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:12:47.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:12:47.484 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:12:47.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:12:47.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:12:47.485 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:12:47.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:47.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:12:47.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:12:47.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:47.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:12:47.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T13:12:47.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-code 2026-02-19T13:12:47.489 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:12:47.489 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:47.489 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.37389 2026-02-19T13:12:47.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.37389 2026-02-19T13:12:47.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:12:47.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:12:47.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T13:12:47.491 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T13:12:47.492 DEBUG:teuthology.orchestra.run.trial111:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T13:12:47.519 INFO:tasks.workunit:Running workunit erasure-code/test-erasure-eio.sh... 2026-02-19T13:12:47.519 DEBUG:teuthology.orchestra.run.trial111:workunit test erasure-code/test-erasure-eio.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=624b9a7bcb2064ce17206482df6a1da007ff7675 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh 2026-02-19T13:12:47.584 INFO:tasks.workunit.client.0.trial111.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-19T13:12:47.590 INFO:tasks.workunit.client.0.trial111.stderr:+ PS4='${BASH_SOURCE[0]}:$LINENO: ${FUNCNAME[0]}: ' 2026-02-19T13:12:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: export PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T13:12:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2394: main: PATH=.:/home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin 2026-02-19T13:12:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: export PYTHONWARNINGS=ignore 2026-02-19T13:12:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2395: main: PYTHONWARNINGS=ignore 2026-02-19T13:12:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: export CEPH_CONF=/dev/null 2026-02-19T13:12:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2396: main: CEPH_CONF=/dev/null 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2397: main: unset CEPH_ARGS 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2399: main: local code 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2400: main: run td/test-erasure-eio 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:22: run: local dir=td/test-erasure-eio 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:23: run: shift 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: export CEPH_MON=127.0.0.1:7112 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:25: run: CEPH_MON=127.0.0.1:7112 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:26: run: export CEPH_ARGS 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: uuidgen 2026-02-19T13:12:47.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:27: run: CEPH_ARGS+='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none ' 2026-02-19T13:12:47.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:28: run: CEPH_ARGS+='--mon-host=127.0.0.1:7112 ' 2026-02-19T13:12:47.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:29: run: CEPH_ARGS+='--osd_mclock_override_recovery_settings=true ' 2026-02-19T13:12:47.592 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: set 2026-02-19T13:12:47.592 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: sed -n -e 's/^\(TEST_[0-9a-z_]*\) .*/\1/p' 2026-02-19T13:12:47.594 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:31: run: local 'funcs=TEST_ec_backfill_unfound 2026-02-19T13:12:47.594 INFO:tasks.workunit.client.0.trial111.stderr:TEST_ec_object_attr_read_error 2026-02-19T13:12:47.594 INFO:tasks.workunit.client.0.trial111.stderr:TEST_ec_recovery_multiple_errors 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_ec_recovery_multiple_objects 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_ec_recovery_multiple_objects_eio 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_ec_recovery_unfound 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_ec_single_recovery_error 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_get_bad_size_shard_0 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_get_bad_size_shard_1 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_get_subread_eio_shard_0 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_get_subread_eio_shard_1 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_get_subread_missing 2026-02-19T13:12:47.595 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_get_with_subreadall_eio_shard_0 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr:TEST_rados_get_with_subreadall_eio_shard_1' 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:12:47.596 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:12:47.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:12:47.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:12:47.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:12:47.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:12:47.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:12:47.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:12:47.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:12:47.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:12:47.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:12:47.599 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:12:47.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:47.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:12:47.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:12:47.599 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:12:47.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:12:47.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:12:47.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:12:47.600 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:12:47.600 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:47.600 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:47.600 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:12:47.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:12:47.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:12:47.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:12:47.601 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:12:47.601 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:47.602 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:47.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:12:47.602 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:12:47.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 1024 -le 1024 ']' 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:127: setup: ulimit -n 4096 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:12:47.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:12:47.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:12:47.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:47.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:47.623 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:47.624 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:47.624 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:47.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:47.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:12:47.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:12:47.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:12:47.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:12:47.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:12:47.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:12:47.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:12:47.646 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:12:47.646 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:12:47.646 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:12:47.647 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:12:47.647 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:47.647 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:47.647 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:12:47.647 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:12:47.647 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:12:47.706 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:12:47.707 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:12:47.707 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:12:47.707 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:12:47.707 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:12:47.707 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:12:47.707 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:12:47.707 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:12:47.708 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:12:47.708 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:47.708 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:47.708 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:12:47.708 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:12:47.708 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:12:47.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:12:47.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:12:47.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:12:47.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:12:47.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:12:47.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:12:47.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:12:47.882 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:12:47.882 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:47.882 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:47.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:47.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:47.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:47.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:47.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:12:47.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:12:47.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:12:47.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:12:48.004 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:12:48.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:12:49.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:12:49.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:12:49.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:12:49.017 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:12:49.018 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:49.018 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:49.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:12:49.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:12:49.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:12:49.074 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:12:49.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:12:49.082 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:12:47.640+0000 7ff2530a5dc0 0 load: jerasure load: lrc 2026-02-19T13:12:49.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_backfill_unfound td/test-erasure-eio 2026-02-19T13:12:49.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:531: TEST_ec_backfill_unfound: local dir=td/test-erasure-eio 2026-02-19T13:12:49.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:532: TEST_ec_backfill_unfound: local objname=myobject 2026-02-19T13:12:49.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:533: TEST_ec_backfill_unfound: local lastobj=300 2026-02-19T13:12:49.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:535: TEST_ec_backfill_unfound: local testobj=obj250 2026-02-19T13:12:49.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:537: TEST_ec_backfill_unfound: ORIG_ARGS='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:12:49.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:538: TEST_ec_backfill_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:12:49.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:539: TEST_ec_backfill_unfound: setup_osds 5 2026-02-19T13:12:49.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-19T13:12:49.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:12:49.085 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-19T13:12:49.088 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-19T13:12:49.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:12:49.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:12:49.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:12:49.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6224972f-4a14-43da-b8e8-ae5756fe02a8 2026-02-19T13:12:49.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 6224972f-4a14-43da-b8e8-ae5756fe02a8' 2026-02-19T13:12:49.092 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 6224972f-4a14-43da-b8e8-ae5756fe02a8 2026-02-19T13:12:49.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:12:49.099 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBRDJdpnCr8BRAAcMvC+BLtVCP712L5sUc8uQ== 2026-02-19T13:12:49.099 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBRDJdpnCr8BRAAcMvC+BLtVCP712L5sUc8uQ=="}' 2026-02-19T13:12:49.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6224972f-4a14-43da-b8e8-ae5756fe02a8 -i td/test-erasure-eio/0/new.json 2026-02-19T13:12:49.321 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:12:49.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:12:49.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBRDJdpnCr8BRAAcMvC+BLtVCP712L5sUc8uQ== --osd-uuid 6224972f-4a14-43da-b8e8-ae5756fe02a8 2026-02-19T13:12:49.353 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:49.353+0000 7fd0f4dbb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:49.354 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:49.354+0000 7fd0f4dbb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:49.355 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:49.355+0000 7fd0f4dbb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:49.356 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:49.356+0000 7fd0f4dbb900 -1 bdev(0x5574a8642c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:12:49.356 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:49.356+0000 7fd0f4dbb900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:12:49.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:12:49.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:12:49.774 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:12:49.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:12:49.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:12:50.087 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:12:50.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:12:50.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:12:50.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:12:50.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:12:50.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:12:50.103 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:50.104+0000 7fec74161900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:50.105 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:50.105+0000 7fec74161900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:50.106 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:50.106+0000 7fec74161900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:50.290 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:50.291+0000 7fec74161900 -1 Falling back to public interface 2026-02-19T13:12:50.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:12:50.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:12:50.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:12:50.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:12:50.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:12:50.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:50.314 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:12:50.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:12:50.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:50.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:12:50.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:12:50.663 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:50.663+0000 7fec74161900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:12:51.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:12:51.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:51.537 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:12:51.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:12:51.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:51.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:12:51.761 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3467491340,v1:127.0.0.1:6803/3467491340] [v2:127.0.0.1:6804/3467491340,v1:127.0.0.1:6805/3467491340] exists,up 6224972f-4a14-43da-b8e8-ae5756fe02a8 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:51.762 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:12:51.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:12:51.764 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:12:51.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e9b7954d-9a59-4760-a1fa-1a0dbf40ecb2 2026-02-19T13:12:51.764 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 e9b7954d-9a59-4760-a1fa-1a0dbf40ecb2 2026-02-19T13:12:51.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e9b7954d-9a59-4760-a1fa-1a0dbf40ecb2' 2026-02-19T13:12:51.765 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:12:51.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBTDJdp3PdWLhAA82DjUJb5g21h0ItKOekbTw== 2026-02-19T13:12:51.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBTDJdp3PdWLhAA82DjUJb5g21h0ItKOekbTw=="}' 2026-02-19T13:12:51.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e9b7954d-9a59-4760-a1fa-1a0dbf40ecb2 -i td/test-erasure-eio/1/new.json 2026-02-19T13:12:51.996 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:12:52.008 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:12:52.009 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBTDJdp3PdWLhAA82DjUJb5g21h0ItKOekbTw== --osd-uuid e9b7954d-9a59-4760-a1fa-1a0dbf40ecb2 2026-02-19T13:12:52.025 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.026+0000 7f0e7e946900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:52.026 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.027+0000 7f0e7e946900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:52.027 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.028+0000 7f0e7e946900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:52.028 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.028+0000 7f0e7e946900 -1 bdev(0x55f2122a6c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:12:52.028 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.028+0000 7f0e7e946900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:12:52.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:12:52.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:12:52.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:12:52.464 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:12:52.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:12:52.763 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:12:52.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:12:52.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:12:52.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:12:52.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:12:52.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:12:52.780 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.780+0000 7f2fc3673900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:52.781 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.782+0000 7f2fc3673900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:52.782 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.783+0000 7f2fc3673900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:52.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:12:52.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:12:52.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:12:52.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:12:52.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:12:52.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:52.986 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:12:52.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:12:52.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:52.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:12:52.998 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:52.999+0000 7f2fc3673900 -1 Falling back to public interface 2026-02-19T13:12:53.208 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:12:53.365 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:53.365+0000 7f2fc3673900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:12:54.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:12:54.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:54.210 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:12:54.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:12:54.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:54.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:12:54.438 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/405245835,v1:127.0.0.1:6811/405245835] [v2:127.0.0.1:6812/405245835,v1:127.0.0.1:6813/405245835] exists,up e9b7954d-9a59-4760-a1fa-1a0dbf40ecb2 2026-02-19T13:12:54.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:12:54.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:12:54.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:12:54.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:12:54.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:12:54.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:12:54.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:12:54.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:12:54.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:12:54.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:12:54.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:12:54.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:12:54.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:12:54.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:12:54.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:54.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:12:54.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:12:54.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ff14ee32-87e9-4173-95aa-3b73164497bf 2026-02-19T13:12:54.446 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 ff14ee32-87e9-4173-95aa-3b73164497bf 2026-02-19T13:12:54.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 ff14ee32-87e9-4173-95aa-3b73164497bf' 2026-02-19T13:12:54.447 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:12:54.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBWDJdp3TkcGxAAmzB+yv1UHH+yVBdPF63usg== 2026-02-19T13:12:54.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBWDJdp3TkcGxAAmzB+yv1UHH+yVBdPF63usg=="}' 2026-02-19T13:12:54.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ff14ee32-87e9-4173-95aa-3b73164497bf -i td/test-erasure-eio/2/new.json 2026-02-19T13:12:54.675 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:12:54.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:12:54.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBWDJdp3TkcGxAAmzB+yv1UHH+yVBdPF63usg== --osd-uuid ff14ee32-87e9-4173-95aa-3b73164497bf 2026-02-19T13:12:54.705 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:54.705+0000 7f894e5bd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:54.706 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:54.706+0000 7f894e5bd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:54.707 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:54.707+0000 7f894e5bd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:54.707 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:54.708+0000 7f894e5bd900 -1 bdev(0x55e9832b0c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:12:54.707 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:54.708+0000 7f894e5bd900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:12:55.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:12:55.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:12:55.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:12:55.121 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:12:55.121 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:12:55.420 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:12:55.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:12:55.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:12:55.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:12:55.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:12:55.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:12:55.436 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:55.437+0000 7f591366a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:55.437 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:55.439+0000 7f591366a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:55.438 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:55.440+0000 7f591366a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:55.641 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:55.643+0000 7f591366a900 -1 Falling back to public interface 2026-02-19T13:12:55.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:12:55.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:12:55.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:12:55.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:12:55.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:12:55.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:55.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:12:55.642 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:12:55.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:55.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:12:55.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:12:55.968 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:55.969+0000 7f591366a900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:12:56.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:12:56.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:56.865 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:12:56.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:12:56.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:56.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:12:57.093 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1019034014,v1:127.0.0.1:6819/1019034014] [v2:127.0.0.1:6820/1019034014,v1:127.0.0.1:6821/1019034014] exists,up ff14ee32-87e9-4173-95aa-3b73164497bf 2026-02-19T13:12:57.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:12:57.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:12:57.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:12:57.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:12:57.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:12:57.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:12:57.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:12:57.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:12:57.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:12:57.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:12:57.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:12:57.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:12:57.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:12:57.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:12:57.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:12:57.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:12:57.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:12:57.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:12:57.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:12:57.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:12:57.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:57.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:57.104 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:57.104 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:57.105 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:57.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:57.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:57.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:12:57.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:12:57.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:12:57.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:12:57.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:12:57.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:12:57.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:12:57.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:12:57.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:12:57.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:12:57.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:12:57.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:12:57.107 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:12:57.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=48e57162-07bb-415e-be74-97644a5c2ed8 2026-02-19T13:12:57.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 48e57162-07bb-415e-be74-97644a5c2ed8' 2026-02-19T13:12:57.108 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 48e57162-07bb-415e-be74-97644a5c2ed8 2026-02-19T13:12:57.108 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:12:57.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBZDJdp9dmOBhAAj5AAhnmn5N53daEiKWfVlw== 2026-02-19T13:12:57.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBZDJdp9dmOBhAAj5AAhnmn5N53daEiKWfVlw=="}' 2026-02-19T13:12:57.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 48e57162-07bb-415e-be74-97644a5c2ed8 -i td/test-erasure-eio/3/new.json 2026-02-19T13:12:57.338 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:12:57.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:12:57.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBZDJdp9dmOBhAAj5AAhnmn5N53daEiKWfVlw== --osd-uuid 48e57162-07bb-415e-be74-97644a5c2ed8 2026-02-19T13:12:57.367 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:57.368+0000 7f0bb7270900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:57.368 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:57.370+0000 7f0bb7270900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:57.369 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:57.371+0000 7f0bb7270900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:57.369 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:57.371+0000 7f0bb7270900 -1 bdev(0x55af18334c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:12:57.370 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:57.371+0000 7f0bb7270900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:12:57.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:12:57.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:12:57.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:12:57.787 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:12:57.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:12:58.087 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:12:58.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:12:58.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:12:58.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:12:58.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:12:58.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:12:58.103 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:58.104+0000 7f23a91ce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:58.104 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:58.106+0000 7f23a91ce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:58.105 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:58.107+0000 7f23a91ce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:12:58.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:12:58.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:12:58.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:12:58.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:12:58.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:12:58.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:58.308 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:12:58.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:12:58.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:58.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:12:58.325 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:58.326+0000 7f23a91ce900 -1 Falling back to public interface 2026-02-19T13:12:58.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:12:58.745 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:12:58.746+0000 7f23a91ce900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:12:59.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:12:59.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:12:59.530 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:12:59.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:12:59.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:12:59.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:12:59.753 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1981011744,v1:127.0.0.1:6827/1981011744] [v2:127.0.0.1:6828/1981011744,v1:127.0.0.1:6829/1981011744] exists,up 48e57162-07bb-415e-be74-97644a5c2ed8 2026-02-19T13:12:59.754 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:12:59.755 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:12:59.756 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:12:59.756 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:59.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:12:59.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:12:59.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=0cfe1f7b-1988-4206-b9b7-d5c914ca9fc8 2026-02-19T13:12:59.757 INFO:tasks.workunit.client.0.trial111.stdout:add osd4 0cfe1f7b-1988-4206-b9b7-d5c914ca9fc8 2026-02-19T13:12:59.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 0cfe1f7b-1988-4206-b9b7-d5c914ca9fc8' 2026-02-19T13:12:59.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:12:59.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBbDJdpJ2DoLRAA3IQmv4Mz3v2Fg2DPEur6FQ== 2026-02-19T13:12:59.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBbDJdpJ2DoLRAA3IQmv4Mz3v2Fg2DPEur6FQ=="}' 2026-02-19T13:12:59.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 0cfe1f7b-1988-4206-b9b7-d5c914ca9fc8 -i td/test-erasure-eio/4/new.json 2026-02-19T13:12:59.993 INFO:tasks.workunit.client.0.trial111.stdout:4 2026-02-19T13:13:00.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T13:13:00.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBbDJdpJ2DoLRAA3IQmv4Mz3v2Fg2DPEur6FQ== --osd-uuid 0cfe1f7b-1988-4206-b9b7-d5c914ca9fc8 2026-02-19T13:13:00.021 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.023+0000 7f0dd3df9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:00.023 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.024+0000 7f0dd3df9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:00.023 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.025+0000 7f0dd3df9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:00.024 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.025+0000 7f0dd3df9900 -1 bdev(0x55718dff4c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:13:00.024 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.025+0000 7f0dd3df9900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T13:13:00.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T13:13:00.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:13:00.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T13:13:00.446 INFO:tasks.workunit.client.0.trial111.stdout:adding osd4 key to auth repository 2026-02-19T13:13:00.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:13:00.745 INFO:tasks.workunit.client.0.trial111.stdout:start osd.4 2026-02-19T13:13:00.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T13:13:00.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:13:00.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:13:00.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:13:00.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:13:00.762 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.763+0000 7f888f264900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:00.763 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.765+0000 7f888f264900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:00.764 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.766+0000 7f888f264900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:00.950 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:00.951+0000 7f888f264900 -1 Falling back to public interface 2026-02-19T13:13:00.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T13:13:00.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:13:00.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T13:13:00.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:13:00.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:13:00.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:13:00.968 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:13:00.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:13:00.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:13:00.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:13:01.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:13:01.321 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:01.322+0000 7f888f264900 -1 osd.4 0 log_to_monitors true 2026-02-19T13:13:02.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:13:02.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:13:02.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:13:02.190 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:13:02.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:13:02.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:13:02.416 INFO:tasks.workunit.client.0.trial111.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3988394796,v1:127.0.0.1:6835/3988394796] [v2:127.0.0.1:6836/3988394796,v1:127.0.0.1:6837/3988394796] exists,up 0cfe1f7b-1988-4206-b9b7-d5c914ca9fc8 2026-02-19T13:13:02.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:13:02.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:13:02.418 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:13:02.476 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:13:02.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:13:02.485 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:12:50.293+0000 7fec74161900 0 load: jerasure load: lrc 2026-02-19T13:13:02.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:540: TEST_ec_backfill_unfound: CEPH_ARGS='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:13:02.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:542: TEST_ec_backfill_unfound: local poolname=pool-jerasure 2026-02-19T13:13:02.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:543: TEST_ec_backfill_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T13:13:02.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:13:02.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:13:02.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T13:13:02.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:13:02.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T13:13:02.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:13:02.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T13:13:02.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:13:02.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:13:03.133 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:13:03.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:04.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:13:04.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:04.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:04.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:04.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:04.148 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:04.148 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:04.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:04.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:04.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:04.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:04.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:04.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:04.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:04.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:04.224 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:04.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:13:04.451 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:13:04.451 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:13:04.451 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:13:04.451 INFO:tasks.workunit.client.0.trial111.stderr:4' 2026-02-19T13:13:04.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:04.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:04.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:04.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T13:13:04.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T13:13:04.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T13:13:04.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:04.532 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:13:04.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-19T13:13:04.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-19T13:13:04.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-19T13:13:04.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:04.610 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:13:04.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378629 2026-02-19T13:13:04.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378629 2026-02-19T13:13:04.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378629' 2026-02-19T13:13:04.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:04.692 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:13:04.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149700 2026-02-19T13:13:04.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149700 2026-02-19T13:13:04.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378629 3-111669149700' 2026-02-19T13:13:04.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:04.770 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:13:04.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888067 2026-02-19T13:13:04.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888067 2026-02-19T13:13:04.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378629 3-111669149700 4-146028888067' 2026-02-19T13:13:04.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:04.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T13:13:04.849 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:04.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:04.850 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-19T13:13:04.850 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:04.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-19T13:13:04.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T13:13:04.851 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803781 2026-02-19T13:13:04.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:05.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-19T13:13:05.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:06.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:06.072 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:06.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-19T13:13:06.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:07.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:13:07.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:07.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-19T13:13:07.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:07.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-19T13:13:07.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:07.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:13:07.521 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574853 2026-02-19T13:13:07.521 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:07.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574853 2026-02-19T13:13:07.523 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574853 2026-02-19T13:13:07.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-19T13:13:07.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:13:07.742 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-19T13:13:07.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:07.743 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378629 2026-02-19T13:13:07.743 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:07.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:13:07.745 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-81604378629 2026-02-19T13:13:07.745 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:07.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378629 2026-02-19T13:13:07.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 81604378629' 2026-02-19T13:13:07.746 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 81604378629 2026-02-19T13:13:07.747 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:13:07.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378629 -lt 81604378629 2026-02-19T13:13:07.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:07.965 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149700 2026-02-19T13:13:07.965 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:07.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:13:07.967 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149700 2026-02-19T13:13:07.967 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:07.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149700 2026-02-19T13:13:07.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149700' 2026-02-19T13:13:07.969 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149700 2026-02-19T13:13:07.969 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:13:08.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149701 -lt 111669149700 2026-02-19T13:13:08.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:08.191 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888067 2026-02-19T13:13:08.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:08.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:13:08.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888067 2026-02-19T13:13:08.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:08.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888067 2026-02-19T13:13:08.195 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 146028888067 2026-02-19T13:13:08.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888067' 2026-02-19T13:13:08.195 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:13:08.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888068 -lt 146028888067 2026-02-19T13:13:08.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:08.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:08.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:08.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:13:08.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:08.716 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:08.716 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:08.716 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:08.716 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:08.716 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:08.717 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:08.935 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:13:08.936 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:08.936 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:08.936 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:09.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:13:09.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:09.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:09.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:545: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:03.370138+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T13:13:03.076615+0000 0'0 2026-02-19T13:13:03.076615+0000 0 0 periodic scrub scheduled @ 2026-02-20T15:42:04.245864+0000 0 0 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.699450+0000 0'0 41:21 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T17:57:38.147305+0000 0 0 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.305847+0000 0'0 41:97 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:31:08.291712+0000 0 0 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.844698+0000 0'0 41:21 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-21T00:33:08.776033+0000 0 0 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:12:56.680320+0000 0'0 40:62 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-21T01:09:21.765455+0000 0 0 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stdout: 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:13:09.444 INFO:tasks.workunit.client.0.trial111.stderr:dumped pgs 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:547: TEST_ec_backfill_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:13:09.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:09.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: get_primary pool-jerasure myobject 2026-02-19T13:13:09.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-19T13:13:09.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-19T13:13:09.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:13:09.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:13:09.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:548: TEST_ec_backfill_unfound: local primary=3 2026-02-19T13:13:09.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: get_osds pool-jerasure myobject 2026-02-19T13:13:09.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:13:09.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:13:09.698 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:13:09.698 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:13:09.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:13:09.921 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:13:09.921 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:13:09.921 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:13:09.921 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:13:09.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:550: TEST_ec_backfill_unfound: local -a initial_osds 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:551: TEST_ec_backfill_unfound: local last_osd=2 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:552: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:13:09.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:13:10.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:13:10.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:553: TEST_ec_backfill_unfound: ceph osd down 2 2026-02-19T13:13:10.446 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already down. 2026-02-19T13:13:10.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:554: TEST_ec_backfill_unfound: ceph osd out 2 2026-02-19T13:13:10.698 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already out. 2026-02-19T13:13:10.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:556: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-19T13:13:10.918 INFO:tasks.workunit.client.0.trial111.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:13:10.918 INFO:tasks.workunit.client.0.trial111.stdout:2.0 1 0 0 0 0 4096 0 0 1 0 1 active+clean 2026-02-19T13:13:03.370138+0000 41'1 41:17 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T13:13:03.076615+0000 0'0 2026-02-19T13:13:03.076615+0000 0 0 periodic scrub scheduled @ 2026-02-20T15:42:04.245864+0000 0 0 2026-02-19T13:13:10.918 INFO:tasks.workunit.client.0.trial111.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.699450+0000 0'0 41:21 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T17:57:38.147305+0000 0 0 2026-02-19T13:13:10.918 INFO:tasks.workunit.client.0.trial111.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.305847+0000 0'0 41:97 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:31:08.291712+0000 0 0 2026-02-19T13:13:10.918 INFO:tasks.workunit.client.0.trial111.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.844698+0000 0'0 41:21 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-21T00:33:08.776033+0000 0 0 2026-02-19T13:13:10.918 INFO:tasks.workunit.client.0.trial111.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:12:56.680320+0000 0'0 41:64 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-21T01:09:21.765455+0000 0 0 2026-02-19T13:13:10.918 INFO:tasks.workunit.client.0.trial111.stdout: 2026-02-19T13:13:10.919 INFO:tasks.workunit.client.0.trial111.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:13:10.919 INFO:tasks.workunit.client.0.trial111.stderr:dumped pgs 2026-02-19T13:13:10.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:558: TEST_ec_backfill_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-19T13:13:10.930 INFO:tasks.workunit.client.0.trial111.stderr:4+0 records in 2026-02-19T13:13:10.930 INFO:tasks.workunit.client.0.trial111.stderr:4+0 records out 2026-02-19T13:13:10.930 INFO:tasks.workunit.client.0.trial111.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 5.926e-05 s, 69.1 MB/s 2026-02-19T13:13:10.930 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: seq 1 300 2026-02-19T13:13:10.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:10.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:10.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:10.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:10.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:10.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:10.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:10.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.041 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.041 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.076 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.614 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.615 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.720 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.720 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:11.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:11.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.532 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.532 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.672 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.673 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj101 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj102 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj103 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj104 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj105 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj106 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj107 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj108 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj109 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj110 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj111 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj112 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj113 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj114 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj115 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj116 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj117 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:12.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:12.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj118 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.013 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj119 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj120 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj121 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj122 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj123 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj124 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj125 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj126 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.157 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj127 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj128 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj129 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.209 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.209 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj130 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj131 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj132 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj133 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.280 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.280 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj134 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj135 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj136 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj137 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj138 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj139 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj140 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj141 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj142 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj143 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj144 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj145 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj146 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj147 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj148 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj149 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj150 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj151 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj152 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj153 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj154 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj155 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj156 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj157 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj158 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj159 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.741 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.741 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj160 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj161 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj162 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj163 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj164 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj165 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.848 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj166 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj167 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj168 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.901 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.901 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj169 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj170 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj171 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj172 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj173 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:13.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:13.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj174 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.012 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj175 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj176 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj177 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj178 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj179 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj180 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj181 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj182 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj183 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj184 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj185 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.209 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.209 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj186 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj187 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj188 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj189 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj190 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj191 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj192 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj193 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj194 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj195 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj196 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj197 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj198 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj199 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj200 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj201 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj202 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj203 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj204 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.543 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.543 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj205 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.561 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.561 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj206 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj207 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj208 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.615 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.615 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj209 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj210 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj211 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj212 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj213 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj214 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.722 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.722 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj215 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj216 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj217 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj218 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj219 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj220 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj221 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj222 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj223 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj224 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj225 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj226 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj227 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj228 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj229 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:14.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:14.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj230 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.012 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.013 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj231 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj232 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj233 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj234 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj235 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj236 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.121 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.121 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj237 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj238 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj239 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj240 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj241 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj242 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj243 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj244 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj245 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj246 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj247 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj248 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj249 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj250 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj251 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj252 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj253 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj254 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj255 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj256 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj257 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj258 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj259 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj260 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj261 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj262 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj263 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj264 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj265 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj266 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj267 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj268 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj269 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj270 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj271 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj272 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj273 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj274 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.803 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.803 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj275 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.820 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.820 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj276 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.837 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj277 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj278 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj279 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj280 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj281 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj282 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj283 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj284 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:15.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:15.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj285 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj286 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj287 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj288 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj289 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj290 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj291 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj292 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj293 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj294 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj295 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj296 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj297 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj298 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj299 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:559: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:16.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:561: TEST_ec_backfill_unfound: rados --pool pool-jerasure put obj300 td/test-erasure-eio/ORIGINAL 2026-02-19T13:13:16.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:564: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 0 2026-02-19T13:13:16.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:13:16.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:13:16.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:13:16.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj250 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj250 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-19T13:13:16.273 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:13:16.497 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:13:16.497 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:13:16.497 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:13:16.497 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:13:16.497 INFO:tasks.workunit.client.0.trial111.stderr:2147483647' 2026-02-19T13:13:16.497 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-19T13:13:16.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-19T13:13:16.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:13:16.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-19T13:13:16.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:13:16.498 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T13:13:16.500 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:13:16.501 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:13:16.501 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:13:16.501 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:16.501 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:16.501 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:13:16.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:13:16.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:13:16.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:13:16.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:13:16.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.3.asok injectdataerr pool-jerasure obj250 0 2026-02-19T13:13:16.630 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:565: TEST_ec_backfill_unfound: inject_eio ec data pool-jerasure obj250 td/test-erasure-eio 1 2026-02-19T13:13:16.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:13:16.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:13:16.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:13:16.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:13:16.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:13:16.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:13:16.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj250 2026-02-19T13:13:16.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj250 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj250 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj250 2026-02-19T13:13:16.632 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:13:16.851 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:13:16.851 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:13:16.851 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:13:16.852 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:13:16.852 INFO:tasks.workunit.client.0.trial111.stderr:2147483647' 2026-02-19T13:13:16.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-19T13:13:16.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-19T13:13:16.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:13:16.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:13:16.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:13:16.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T13:13:16.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:13:16.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:13:16.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:13:16.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:13:16.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:16.855 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:13:16.856 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:13:16.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:13:16.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:13:16.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:13:16.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:13:16.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:13:16.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:13:16.921 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:13:16.922 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:16.922 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:16.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:13:16.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:13:16.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.1.asok injectdataerr pool-jerasure obj250 1 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:567: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:13:16.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:16.993 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:13:16.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:13:16.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:13:16.995 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:13:16.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:13:16.996 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:13:16.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:13:16.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:13:16.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:13:16.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:13:17.005 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:17.005+0000 7f25206f3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:17.007 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:17.007+0000 7f25206f3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:17.007 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:17.008+0000 7f25206f3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:17.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:13:17.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:13:17.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:13:17.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:13:17.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:13:17.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:13:17.216 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:13:17.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:13:17.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:13:17.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:13:17.236 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:17.236+0000 7f25206f3900 -1 Falling back to public interface 2026-02-19T13:13:17.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:13:17.648 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:17.648+0000 7f25206f3900 -1 osd.2 41 log_to_monitors true 2026-02-19T13:13:18.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:13:18.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:13:18.436 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:13:18.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:13:18.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:13:18.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:13:18.662 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up out weight 0 up_from 47 up_thru 19 down_at 42 last_clean_interval [19,41) [v2:127.0.0.1:6818/1590993308,v1:127.0.0.1:6819/1590993308] [v2:127.0.0.1:6820/1590993308,v1:127.0.0.1:6821/1590993308] exists,up ff14ee32-87e9-4173-95aa-3b73164497bf 2026-02-19T13:13:18.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:13:18.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:13:18.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:13:18.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:568: TEST_ec_backfill_unfound: ceph osd in 2 2026-02-19T13:13:18.907 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already in. 2026-02-19T13:13:18.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:570: TEST_ec_backfill_unfound: sleep 15 2026-02-19T13:13:33.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: seq 1 240 2026-02-19T13:13:33.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:572: TEST_ec_backfill_unfound: for tmp in $(seq 1 240) 2026-02-19T13:13:33.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: get_state 2.0 2026-02-19T13:13:33.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-19T13:13:33.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-19T13:13:33.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-19T13:13:33.924 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-19T13:13:34.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:573: TEST_ec_backfill_unfound: state=active+backfill_unfound+undersized+degraded+remapped 2026-02-19T13:13:34.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: echo active+backfill_unfound+undersized+degraded+remapped 2026-02-19T13:13:34.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:574: TEST_ec_backfill_unfound: grep backfill_unfound 2026-02-19T13:13:34.144 INFO:tasks.workunit.client.0.trial111.stdout:active+backfill_unfound+undersized+degraded+remapped 2026-02-19T13:13:34.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:575: TEST_ec_backfill_unfound: '[' 0 = 0 ']' 2026-02-19T13:13:34.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:576: TEST_ec_backfill_unfound: break 2026-02-19T13:13:34.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:582: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-19T13:13:34.349 INFO:tasks.workunit.client.0.trial111.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:13:34.350 INFO:tasks.workunit.client.0.trial111.stdout:2.0 301 1 28 0 1 1232896 0 0 101 200 101 active+backfill_unfound+undersized+degraded+remapped 2026-02-19T13:13:19.895801+0000 45'301 54:909 [3,1,4,0,2] 3 [3,1,4,0,NONE] 3 0'0 2026-02-19T13:13:03.076615+0000 0'0 2026-02-19T13:13:03.076615+0000 0 0 no scrub is scheduled 0 0 2026-02-19T13:13:34.350 INFO:tasks.workunit.client.0.trial111.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:18.952474+0000 0'0 54:63 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:01:23.745849+0000 0 0 2026-02-19T13:13:34.350 INFO:tasks.workunit.client.0.trial111.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.305847+0000 0'0 54:121 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:31:08.291712+0000 0 0 2026-02-19T13:13:34.350 INFO:tasks.workunit.client.0.trial111.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.844698+0000 0'0 54:45 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-21T00:33:08.776033+0000 0 0 2026-02-19T13:13:34.350 INFO:tasks.workunit.client.0.trial111.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:18.952340+0000 0'0 54:105 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T17:29:25.384388+0000 0 0 2026-02-19T13:13:34.350 INFO:tasks.workunit.client.0.trial111.stdout: 2026-02-19T13:13:34.350 INFO:tasks.workunit.client.0.trial111.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:13:34.351 INFO:tasks.workunit.client.0.trial111.stderr:dumped pgs 2026-02-19T13:13:34.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:583: TEST_ec_backfill_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:13:34.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:13:34.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:13:34.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:13:34.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:13:34.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:13:34.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:13:34.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:584: TEST_ec_backfill_unfound: sleep 5 2026-02-19T13:13:39.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:586: TEST_ec_backfill_unfound: ceph pg dump pgs 2026-02-19T13:13:39.877 INFO:tasks.workunit.client.0.trial111.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:13:39.877 INFO:tasks.workunit.client.0.trial111.stdout:2.0 301 1 303 0 1 1232896 0 0 101 200 101 active+recovery_unfound+undersized+degraded 2026-02-19T13:13:34.524259+0000 45'301 56:921 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-19T13:13:03.076615+0000 0'0 2026-02-19T13:13:03.076615+0000 0 0 no scrub is scheduled 0 0 2026-02-19T13:13:39.877 INFO:tasks.workunit.client.0.trial111.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+undersized 2026-02-19T13:13:34.522847+0000 0'0 56:71 [4,1] 4 [4,1] 4 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T17:29:18.574910+0000 0 0 2026-02-19T13:13:39.877 INFO:tasks.workunit.client.0.trial111.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.305847+0000 0'0 56:125 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-20T19:31:08.291712+0000 0 0 2026-02-19T13:13:39.877 INFO:tasks.workunit.client.0.trial111.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:13:02.844698+0000 0'0 56:49 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 periodic scrub scheduled @ 2026-02-21T00:33:08.776033+0000 0 0 2026-02-19T13:13:39.877 INFO:tasks.workunit.client.0.trial111.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 peering 2026-02-19T13:13:34.418156+0000 0'0 54:106 [1,0] 1 [1,0] 1 0'0 2026-02-19T13:12:47.989563+0000 0'0 2026-02-19T13:12:47.989563+0000 0 0 no scrub is scheduled 0 0 2026-02-19T13:13:39.878 INFO:tasks.workunit.client.0.trial111.stdout: 2026-02-19T13:13:39.878 INFO:tasks.workunit.client.0.trial111.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:13:39.878 INFO:tasks.workunit.client.0.trial111.stderr:dumped pgs 2026-02-19T13:13:39.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:587: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:13:39.960 INFO:tasks.workunit.client.0.trial111.stdout:{ 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "num_missing": 1, 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "num_unfound": 1, 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "objects": [ 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "oid": { 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "oid": "obj250", 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "key": "", 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "snapid": -2, 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "hash": 2249616407, 2026-02-19T13:13:39.961 INFO:tasks.workunit.client.0.trial111.stdout: "max": 0, 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "pool": 2, 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "namespace": "" 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "need": "45'251", 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "have": "0'0", 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "flags": "none", 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "locations": [ 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: ] 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:39.962 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: "state": "NotRecovering", 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: "available_might_have_unfound": true, 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: "might_have_unfound": [ 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "2(4)", 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: "status": "osd is down" 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout: "more": false 2026-02-19T13:13:39.963 INFO:tasks.workunit.client.0.trial111.stdout:} 2026-02-19T13:13:39.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:588: TEST_ec_backfill_unfound: ceph pg 2.0 query 2026-02-19T13:13:40.043 INFO:tasks.workunit.client.0.trial111.stdout:{ 2026-02-19T13:13:40.043 INFO:tasks.workunit.client.0.trial111.stdout: "snap_trimq": "[]", 2026-02-19T13:13:40.043 INFO:tasks.workunit.client.0.trial111.stdout: "snap_trimq_len": 0, 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: "epoch": 56, 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:13:40.044 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: "acting_recovery_backfill": [ 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: "1(1)", 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: "3(0)", 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: "info": { 2026-02-19T13:13:40.045 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s0", 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "0", 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'301", 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'250", 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "45'200", 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 301, 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "55", 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:13:40.046 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:13:40.047 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 56, 2026-02-19T13:13:40.047 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 55, 2026-02-19T13:13:40.047 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:13:40.047 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:13:40.047 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:13:40.047 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:13:40.047 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 55, 2026-02-19T13:13:40.047 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 55, 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'301", 2026-02-19T13:13:40.048 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 922, 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 56, 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+recovery_unfound+undersized+degraded", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:13:34.524259+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:13:34.524259+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:13:34.524259+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:13:34.524259+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:13:09.474722+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:13:34.522754+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:13:34.522754+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:13:34.524259+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:13:34.521507+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:13:34.521424+0000", 2026-02-19T13:13:40.049 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 55, 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "45'200", 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "45'200", 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.050 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.051 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:13:40.051 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:13:40.051 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 200, 2026-02-19T13:13:40.051 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:13:40.051 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:13:40.051 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:13:40.051 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:13:40.051 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 1232896, 2026-02-19T13:13:40.052 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 301, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 1505, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 1, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 1, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 303, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 1, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 301, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 301, 2026-02-19T13:13:40.053 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 1204, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:13:40.054 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:13:40.055 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:13:40.056 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.056 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:13:40.056 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.056 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.056 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.056 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.056 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.056 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [ 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [ 2026-02-19T13:13:40.057 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.058 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T13:13:40.058 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 300 2026-02-19T13:13:40.058 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.058 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.058 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),4(2)", 2026-02-19T13:13:40.058 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 1 2026-02-19T13:13:40.058 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.058 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 56, 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:13:40.059 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "peer_info": [ 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "peer": "0(3)", 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s3", 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "3", 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'301", 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'301", 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "45'200", 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 301, 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:13:40.060 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 56, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 55, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 55, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 55, 2026-02-19T13:13:40.061 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'301", 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 912, 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 55, 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+undersized+degraded", 2026-02-19T13:13:40.062 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:13:09.474722+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:13:10.087150+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:13:10.087093+0000", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 55, 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "45'200", 2026-02-19T13:13:40.063 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "45'200", 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 200, 2026-02-19T13:13:40.064 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T20:01:37.236398+0000", 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:13:40.065 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 1232896, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 301, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 1505, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 0, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 301, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 0, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 301, 2026-02-19T13:13:40.066 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:13:40.067 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:13:40.067 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:13:40.067 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 301, 2026-02-19T13:13:40.067 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 1204, 2026-02-19T13:13:40.067 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:13:40.067 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:13:40.068 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:13:40.069 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.070 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [ 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: "3(0)", 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: "1(1)", 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [ 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T13:13:40.071 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 301 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:13:40.072 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 56, 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "peer": "1(1)", 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s1", 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "1", 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'301", 2026-02-19T13:13:40.073 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'301", 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "45'200", 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 301, 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 56, 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 55, 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:13:40.074 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 55, 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 55, 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.075 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'301", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 331, 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 45, 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+undersized+degraded", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:13:09.474722+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:13:10.087150+0000", 2026-02-19T13:13:40.076 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:13:10.087093+0000", 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 55, 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "45'200", 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "45'200", 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:13:40.077 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 200, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:13:40.078 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T20:01:37.236398+0000", 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 1232896, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 301, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 1505, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 1, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 301, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:13:40.079 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 301, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 301, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 1204, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:13:40.080 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:13:40.081 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.082 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [ 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "3(0)", 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "1(1)", 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [ 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 301 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:13:40.083 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 56, 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "peer": "4(2)", 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s2", 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "2", 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'301", 2026-02-19T13:13:40.084 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'301", 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "45'200", 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 301, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 56, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 55, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 55, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 55, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:13:40.085 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'301", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 331, 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 45, 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+undersized+degraded", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:13:09.474722+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.086 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:13:10.088185+0000", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:13:16.272318+0000", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:13:10.087150+0000", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:13:10.087093+0000", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 55, 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "45'200", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "45'200", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:13:03.076615+0000", 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:13:40.087 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 200, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T20:01:37.236398+0000", 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 1232896, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 301, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 1505, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:13:40.088 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 301, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 301, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 301, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 1204, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:13:40.089 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.090 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: 2147483647 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [ 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: "3(0)", 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: "1(1)", 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:13:40.091 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [ 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 301 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 56, 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.092 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: "recovery_state": [ 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: "name": "Started/Primary/Active", 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: "enter_time": "2026-02-19T13:13:34.521434+0000", 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: "might_have_unfound": [ 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "0(3)", 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: "status": "already probed" 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "1(1)", 2026-02-19T13:13:40.093 INFO:tasks.workunit.client.0.trial111.stdout: "status": "already probed" 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "2(4)", 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "status": "osd is down" 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "4(2)", 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "status": "already probed" 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "recovery_progress": { 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "backfill_targets": [], 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "waiting_on_backfill": [], 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill_started": "MIN", 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "backfill_info": { 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "begin": "MIN", 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "end": "MIN", 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "objects": [] 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.094 INFO:tasks.workunit.client.0.trial111.stdout: "peer_backfill_info": [], 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "backfills_in_flight": [], 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "recovering": [], 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "pg_backend": { 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "recovery_ops": [], 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "read_ops": [] 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "name": "Started", 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "enter_time": "2026-02-19T13:13:34.418596+0000" 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "scrubber": { 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "active": false, 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "must_scrub": false, 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "must_deep_scrub": false, 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "must_repair": false, 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_reg_stamp": "2026-02-20T21:59:40.230431+0000", 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "schedule": "not registered for scrubbing" 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:13:40.095 INFO:tasks.workunit.client.0.trial111.stdout: "agent_state": {} 2026-02-19T13:13:40.096 INFO:tasks.workunit.client.0.trial111.stdout:} 2026-02-19T13:13:40.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:13:40.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:590: TEST_ec_backfill_unfound: grep -q obj250 2026-02-19T13:13:40.136 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:13:40.136 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: jq .available_might_have_unfound 2026-02-19T13:13:40.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:592: TEST_ec_backfill_unfound: check=true 2026-02-19T13:13:40.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:593: TEST_ec_backfill_unfound: test true == true 2026-02-19T13:13:40.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:13:40.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].status' 2026-02-19T13:13:40.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: eval 'check="osd' is 'down"' 2026-02-19T13:13:40.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:595: TEST_ec_backfill_unfound: check='osd is down' 2026-02-19T13:13:40.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:596: TEST_ec_backfill_unfound: test 'osd is down' == 'osd is down' 2026-02-19T13:13:40.298 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:13:40.298 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: jq '.might_have_unfound[0].osd' 2026-02-19T13:13:40.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: eval 'check="2(4)"' 2026-02-19T13:13:40.381 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:598: TEST_ec_backfill_unfound: check='2(4)' 2026-02-19T13:13:40.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:599: TEST_ec_backfill_unfound: test '2(4)' == '2(4)' 2026-02-19T13:13:40.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:601: TEST_ec_backfill_unfound: activate_osd td/test-erasure-eio 2 2026-02-19T13:13:40.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:13:40.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:13:40.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:13:40.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:13:40.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:13:40.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:13:40.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:13:40.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:13:40.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:40.383 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:13:40.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:13:40.385 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:13:40.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:13:40.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:13:40.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:13:40.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:13:40.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:13:40.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:13:40.394 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:40.395+0000 7ff925440900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:40.396 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:40.396+0000 7ff925440900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:40.397 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:40.397+0000 7ff925440900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:40.554 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:40.554+0000 7ff925440900 -1 Falling back to public interface 2026-02-19T13:13:40.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:13:40.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:13:40.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:13:40.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:13:40.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:13:40.603 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:13:40.603 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:13:40.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:13:40.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:13:40.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:13:40.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:13:40.901 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:40.901+0000 7ff925440900 -1 osd.2 54 log_to_monitors true 2026-02-19T13:13:41.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:13:41.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:13:41.823 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:13:41.824 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:13:41.824 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:13:41.824 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:13:42.046 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 58 up_thru 19 down_at 55 last_clean_interval [47,54) [v2:127.0.0.1:6818/1499208265,v1:127.0.0.1:6819/1499208265] [v2:127.0.0.1:6820/1499208265,v1:127.0.0.1:6821/1499208265] exists,up ff14ee32-87e9-4173-95aa-3b73164497bf 2026-02-19T13:13:42.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:13:42.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:13:42.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:13:42.048 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:604: TEST_ec_backfill_unfound: timeout 5 rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-19T13:13:47.049 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:605: TEST_ec_backfill_unfound: test 124 = 124 2026-02-19T13:13:47.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:607: TEST_ec_backfill_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-19T13:13:47.124 INFO:tasks.workunit.client.0.trial111.stderr:pg has no unfound objects 2026-02-19T13:13:47.134 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:609: TEST_ec_backfill_unfound: wait_for_clean 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:13:47.135 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:13:47.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:13:47.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:13:47.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:13:47.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:13:47.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:13:47.210 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:13:47.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:13:47.431 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:13:47.431 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:13:47.431 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:13:47.431 INFO:tasks.workunit.client.0.trial111.stderr:4' 2026-02-19T13:13:47.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:13:47.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:47.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:13:47.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803792 2026-02-19T13:13:47.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803792 2026-02-19T13:13:47.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803792' 2026-02-19T13:13:47.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:47.513 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:13:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574863 2026-02-19T13:13:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574863 2026-02-19T13:13:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803792 1-55834574863' 2026-02-19T13:13:47.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:47.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:13:47.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=249108103172 2026-02-19T13:13:47.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 249108103172 2026-02-19T13:13:47.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-249108103172' 2026-02-19T13:13:47.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:47.668 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:13:47.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149711 2026-02-19T13:13:47.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149711 2026-02-19T13:13:47.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-249108103172 3-111669149711' 2026-02-19T13:13:47.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:13:47.748 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:13:47.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888078 2026-02-19T13:13:47.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888078 2026-02-19T13:13:47.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803792 1-55834574863 2-249108103172 3-111669149711 4-146028888078' 2026-02-19T13:13:47.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:47.826 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803792 2026-02-19T13:13:47.826 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:47.827 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:13:47.827 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803792 2026-02-19T13:13:47.827 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:47.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803792 2026-02-19T13:13:47.828 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803792 2026-02-19T13:13:47.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803792' 2026-02-19T13:13:47.829 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:48.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803789 -lt 25769803792 2026-02-19T13:13:48.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:13:49.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:13:49.053 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:13:49.280 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803792 -lt 25769803792 2026-02-19T13:13:49.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:49.281 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574863 2026-02-19T13:13:49.281 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:49.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:13:49.282 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574863 2026-02-19T13:13:49.283 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:49.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574863 2026-02-19T13:13:49.284 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574863 2026-02-19T13:13:49.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574863' 2026-02-19T13:13:49.284 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:13:49.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574863 -lt 55834574863 2026-02-19T13:13:49.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:49.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-249108103172 2026-02-19T13:13:49.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:49.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:13:49.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-249108103172 2026-02-19T13:13:49.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:49.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=249108103172 2026-02-19T13:13:49.510 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 249108103172 2026-02-19T13:13:49.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 249108103172' 2026-02-19T13:13:49.511 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:13:49.730 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 249108103172 -lt 249108103172 2026-02-19T13:13:49.730 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:49.731 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149711 2026-02-19T13:13:49.731 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:49.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:13:49.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149711 2026-02-19T13:13:49.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:49.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149711 2026-02-19T13:13:49.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149711' 2026-02-19T13:13:49.734 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149711 2026-02-19T13:13:49.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:13:49.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149711 -lt 111669149711 2026-02-19T13:13:49.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:13:49.956 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888078 2026-02-19T13:13:49.956 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:13:49.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:13:49.957 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888078 2026-02-19T13:13:49.958 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:13:49.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888078 2026-02-19T13:13:49.959 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 146028888078 2026-02-19T13:13:49.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888078' 2026-02-19T13:13:49.959 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:13:50.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888078 -lt 146028888078 2026-02-19T13:13:50.183 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:13:50.184 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:50.184 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:50.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:13:50.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:13:50.483 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:13:50.483 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:13:50.483 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:13:50.483 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:13:50.483 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:13:50.483 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:13:50.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:13:50.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:13:50.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:13:50.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:13:50.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:13:50.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:13:50.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:13:50.995 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: seq 1 300 2026-02-19T13:13:50.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:50.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj1 = obj250 ']' 2026-02-19T13:13:50.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.013 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj2 = obj250 ']' 2026-02-19T13:13:51.015 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj3 = obj250 ']' 2026-02-19T13:13:51.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj4 = obj250 ']' 2026-02-19T13:13:51.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj5 = obj250 ']' 2026-02-19T13:13:51.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj6 = obj250 ']' 2026-02-19T13:13:51.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj7 = obj250 ']' 2026-02-19T13:13:51.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.122 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj8 = obj250 ']' 2026-02-19T13:13:51.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj9 = obj250 ']' 2026-02-19T13:13:51.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.159 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj10 = obj250 ']' 2026-02-19T13:13:51.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj11 = obj250 ']' 2026-02-19T13:13:51.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj12 = obj250 ']' 2026-02-19T13:13:51.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj13 = obj250 ']' 2026-02-19T13:13:51.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj14 = obj250 ']' 2026-02-19T13:13:51.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj15 = obj250 ']' 2026-02-19T13:13:51.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj16 = obj250 ']' 2026-02-19T13:13:51.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj17 = obj250 ']' 2026-02-19T13:13:51.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj18 = obj250 ']' 2026-02-19T13:13:51.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj19 = obj250 ']' 2026-02-19T13:13:51.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj20 = obj250 ']' 2026-02-19T13:13:51.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.358 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.358 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj21 = obj250 ']' 2026-02-19T13:13:51.358 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj22 = obj250 ']' 2026-02-19T13:13:51.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj23 = obj250 ']' 2026-02-19T13:13:51.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj24 = obj250 ']' 2026-02-19T13:13:51.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj25 = obj250 ']' 2026-02-19T13:13:51.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj26 = obj250 ']' 2026-02-19T13:13:51.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj27 = obj250 ']' 2026-02-19T13:13:51.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj28 = obj250 ']' 2026-02-19T13:13:51.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj29 = obj250 ']' 2026-02-19T13:13:51.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj30 = obj250 ']' 2026-02-19T13:13:51.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj31 = obj250 ']' 2026-02-19T13:13:51.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj32 = obj250 ']' 2026-02-19T13:13:51.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.573 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj33 = obj250 ']' 2026-02-19T13:13:51.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj34 = obj250 ']' 2026-02-19T13:13:51.593 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.611 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.611 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj35 = obj250 ']' 2026-02-19T13:13:51.611 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj36 = obj250 ']' 2026-02-19T13:13:51.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj37 = obj250 ']' 2026-02-19T13:13:51.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj38 = obj250 ']' 2026-02-19T13:13:51.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj39 = obj250 ']' 2026-02-19T13:13:51.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj40 = obj250 ']' 2026-02-19T13:13:51.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj41 = obj250 ']' 2026-02-19T13:13:51.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj42 = obj250 ']' 2026-02-19T13:13:51.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj43 = obj250 ']' 2026-02-19T13:13:51.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj44 = obj250 ']' 2026-02-19T13:13:51.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj45 = obj250 ']' 2026-02-19T13:13:51.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.805 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj46 = obj250 ']' 2026-02-19T13:13:51.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj47 = obj250 ']' 2026-02-19T13:13:51.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj48 = obj250 ']' 2026-02-19T13:13:51.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj49 = obj250 ']' 2026-02-19T13:13:51.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj50 = obj250 ']' 2026-02-19T13:13:51.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.894 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj51 = obj250 ']' 2026-02-19T13:13:51.894 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj52 = obj250 ']' 2026-02-19T13:13:51.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj53 = obj250 ']' 2026-02-19T13:13:51.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj54 = obj250 ']' 2026-02-19T13:13:51.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj55 = obj250 ']' 2026-02-19T13:13:51.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:51.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:51.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj56 = obj250 ']' 2026-02-19T13:13:51.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-19T13:13:51.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj57 = obj250 ']' 2026-02-19T13:13:52.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj58 = obj250 ']' 2026-02-19T13:13:52.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj59 = obj250 ']' 2026-02-19T13:13:52.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj60 = obj250 ']' 2026-02-19T13:13:52.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj61 = obj250 ']' 2026-02-19T13:13:52.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj62 = obj250 ']' 2026-02-19T13:13:52.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj63 = obj250 ']' 2026-02-19T13:13:52.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj64 = obj250 ']' 2026-02-19T13:13:52.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj65 = obj250 ']' 2026-02-19T13:13:52.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj66 = obj250 ']' 2026-02-19T13:13:52.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj67 = obj250 ']' 2026-02-19T13:13:52.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj68 = obj250 ']' 2026-02-19T13:13:52.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj69 = obj250 ']' 2026-02-19T13:13:52.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj70 = obj250 ']' 2026-02-19T13:13:52.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj71 = obj250 ']' 2026-02-19T13:13:52.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj72 = obj250 ']' 2026-02-19T13:13:52.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj73 = obj250 ']' 2026-02-19T13:13:52.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj74 = obj250 ']' 2026-02-19T13:13:52.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj75 = obj250 ']' 2026-02-19T13:13:52.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj76 = obj250 ']' 2026-02-19T13:13:52.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj77 = obj250 ']' 2026-02-19T13:13:52.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj78 = obj250 ']' 2026-02-19T13:13:52.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj79 = obj250 ']' 2026-02-19T13:13:52.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj80 = obj250 ']' 2026-02-19T13:13:52.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj81 = obj250 ']' 2026-02-19T13:13:52.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj82 = obj250 ']' 2026-02-19T13:13:52.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj83 = obj250 ']' 2026-02-19T13:13:52.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj84 = obj250 ']' 2026-02-19T13:13:52.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj85 = obj250 ']' 2026-02-19T13:13:52.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj86 = obj250 ']' 2026-02-19T13:13:52.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj87 = obj250 ']' 2026-02-19T13:13:52.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj88 = obj250 ']' 2026-02-19T13:13:52.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj89 = obj250 ']' 2026-02-19T13:13:52.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj90 = obj250 ']' 2026-02-19T13:13:52.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj91 = obj250 ']' 2026-02-19T13:13:52.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj92 = obj250 ']' 2026-02-19T13:13:52.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj93 = obj250 ']' 2026-02-19T13:13:52.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj94 = obj250 ']' 2026-02-19T13:13:52.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.676 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj95 = obj250 ']' 2026-02-19T13:13:52.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj96 = obj250 ']' 2026-02-19T13:13:52.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj97 = obj250 ']' 2026-02-19T13:13:52.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.730 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj98 = obj250 ']' 2026-02-19T13:13:52.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj99 = obj250 ']' 2026-02-19T13:13:52.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj100 = obj250 ']' 2026-02-19T13:13:52.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj101 = obj250 ']' 2026-02-19T13:13:52.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj101 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.803 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.803 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj102 = obj250 ']' 2026-02-19T13:13:52.803 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj102 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.820 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj103 = obj250 ']' 2026-02-19T13:13:52.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj103 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj104 = obj250 ']' 2026-02-19T13:13:52.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj104 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj105 = obj250 ']' 2026-02-19T13:13:52.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj105 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj106 = obj250 ']' 2026-02-19T13:13:52.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj106 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.894 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj107 = obj250 ']' 2026-02-19T13:13:52.894 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj107 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj108 = obj250 ']' 2026-02-19T13:13:52.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj108 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj109 = obj250 ']' 2026-02-19T13:13:52.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj109 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.946 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj110 = obj250 ']' 2026-02-19T13:13:52.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj110 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj111 = obj250 ']' 2026-02-19T13:13:52.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj111 td/test-erasure-eio/CHECK 2026-02-19T13:13:52.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:52.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:52.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj112 = obj250 ']' 2026-02-19T13:13:52.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj112 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj113 = obj250 ']' 2026-02-19T13:13:53.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj113 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj114 = obj250 ']' 2026-02-19T13:13:53.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj114 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj115 = obj250 ']' 2026-02-19T13:13:53.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj115 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj116 = obj250 ']' 2026-02-19T13:13:53.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj116 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj117 = obj250 ']' 2026-02-19T13:13:53.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj117 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj118 = obj250 ']' 2026-02-19T13:13:53.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj118 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj119 = obj250 ']' 2026-02-19T13:13:53.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj119 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj120 = obj250 ']' 2026-02-19T13:13:53.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj120 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj121 = obj250 ']' 2026-02-19T13:13:53.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj121 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.159 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj122 = obj250 ']' 2026-02-19T13:13:53.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj122 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj123 = obj250 ']' 2026-02-19T13:13:53.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj123 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj124 = obj250 ']' 2026-02-19T13:13:53.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj124 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj125 = obj250 ']' 2026-02-19T13:13:53.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj125 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj126 = obj250 ']' 2026-02-19T13:13:53.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj126 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj127 = obj250 ']' 2026-02-19T13:13:53.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj127 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj128 = obj250 ']' 2026-02-19T13:13:53.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj128 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj129 = obj250 ']' 2026-02-19T13:13:53.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj129 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj130 = obj250 ']' 2026-02-19T13:13:53.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj130 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj131 = obj250 ']' 2026-02-19T13:13:53.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj131 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj132 = obj250 ']' 2026-02-19T13:13:53.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj132 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj133 = obj250 ']' 2026-02-19T13:13:53.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj133 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj134 = obj250 ']' 2026-02-19T13:13:53.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj134 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj135 = obj250 ']' 2026-02-19T13:13:53.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj135 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj136 = obj250 ']' 2026-02-19T13:13:53.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj136 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj137 = obj250 ']' 2026-02-19T13:13:53.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj137 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj138 = obj250 ']' 2026-02-19T13:13:53.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj138 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj139 = obj250 ']' 2026-02-19T13:13:53.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj139 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj140 = obj250 ']' 2026-02-19T13:13:53.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj140 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj141 = obj250 ']' 2026-02-19T13:13:53.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj141 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj142 = obj250 ']' 2026-02-19T13:13:53.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj142 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj143 = obj250 ']' 2026-02-19T13:13:53.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj143 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj144 = obj250 ']' 2026-02-19T13:13:53.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj144 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj145 = obj250 ']' 2026-02-19T13:13:53.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj145 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj146 = obj250 ']' 2026-02-19T13:13:53.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj146 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj147 = obj250 ']' 2026-02-19T13:13:53.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj147 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj148 = obj250 ']' 2026-02-19T13:13:53.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj148 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj149 = obj250 ']' 2026-02-19T13:13:53.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj149 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj150 = obj250 ']' 2026-02-19T13:13:53.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj150 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.676 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj151 = obj250 ']' 2026-02-19T13:13:53.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj151 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.693 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj152 = obj250 ']' 2026-02-19T13:13:53.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj152 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj153 = obj250 ']' 2026-02-19T13:13:53.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj153 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.730 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.730 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj154 = obj250 ']' 2026-02-19T13:13:53.730 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj154 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj155 = obj250 ']' 2026-02-19T13:13:53.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj155 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj156 = obj250 ']' 2026-02-19T13:13:53.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj156 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj157 = obj250 ']' 2026-02-19T13:13:53.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj157 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.802 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.802 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj158 = obj250 ']' 2026-02-19T13:13:53.802 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj158 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj159 = obj250 ']' 2026-02-19T13:13:53.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj159 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.837 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj160 = obj250 ']' 2026-02-19T13:13:53.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj160 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj161 = obj250 ']' 2026-02-19T13:13:53.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj161 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj162 = obj250 ']' 2026-02-19T13:13:53.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj162 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj163 = obj250 ']' 2026-02-19T13:13:53.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj163 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj164 = obj250 ']' 2026-02-19T13:13:53.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj164 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj165 = obj250 ']' 2026-02-19T13:13:53.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj165 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.946 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.946 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj166 = obj250 ']' 2026-02-19T13:13:53.946 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj166 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj167 = obj250 ']' 2026-02-19T13:13:53.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj167 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:53.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:53.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj168 = obj250 ']' 2026-02-19T13:13:53.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj168 td/test-erasure-eio/CHECK 2026-02-19T13:13:53.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj169 = obj250 ']' 2026-02-19T13:13:54.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj169 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj170 = obj250 ']' 2026-02-19T13:13:54.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj170 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj171 = obj250 ']' 2026-02-19T13:13:54.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj171 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj172 = obj250 ']' 2026-02-19T13:13:54.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj172 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj173 = obj250 ']' 2026-02-19T13:13:54.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj173 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj174 = obj250 ']' 2026-02-19T13:13:54.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj174 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj175 = obj250 ']' 2026-02-19T13:13:54.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj175 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj176 = obj250 ']' 2026-02-19T13:13:54.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj176 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj177 = obj250 ']' 2026-02-19T13:13:54.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj177 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj178 = obj250 ']' 2026-02-19T13:13:54.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj178 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj179 = obj250 ']' 2026-02-19T13:13:54.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj179 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj180 = obj250 ']' 2026-02-19T13:13:54.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj180 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj181 = obj250 ']' 2026-02-19T13:13:54.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj181 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj182 = obj250 ']' 2026-02-19T13:13:54.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj182 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj183 = obj250 ']' 2026-02-19T13:13:54.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj183 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj184 = obj250 ']' 2026-02-19T13:13:54.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj184 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj185 = obj250 ']' 2026-02-19T13:13:54.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj185 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj186 = obj250 ']' 2026-02-19T13:13:54.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj186 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj187 = obj250 ']' 2026-02-19T13:13:54.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj187 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj188 = obj250 ']' 2026-02-19T13:13:54.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj188 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.355 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj189 = obj250 ']' 2026-02-19T13:13:54.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj189 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj190 = obj250 ']' 2026-02-19T13:13:54.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj190 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj191 = obj250 ']' 2026-02-19T13:13:54.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj191 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj192 = obj250 ']' 2026-02-19T13:13:54.410 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj192 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj193 = obj250 ']' 2026-02-19T13:13:54.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj193 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj194 = obj250 ']' 2026-02-19T13:13:54.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj194 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj195 = obj250 ']' 2026-02-19T13:13:54.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj195 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj196 = obj250 ']' 2026-02-19T13:13:54.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj196 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj197 = obj250 ']' 2026-02-19T13:13:54.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj197 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj198 = obj250 ']' 2026-02-19T13:13:54.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj198 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj199 = obj250 ']' 2026-02-19T13:13:54.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj199 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj200 = obj250 ']' 2026-02-19T13:13:54.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj200 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj201 = obj250 ']' 2026-02-19T13:13:54.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj201 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj202 = obj250 ']' 2026-02-19T13:13:54.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj202 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj203 = obj250 ']' 2026-02-19T13:13:54.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj203 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj204 = obj250 ']' 2026-02-19T13:13:54.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj204 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj205 = obj250 ']' 2026-02-19T13:13:54.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj205 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.658 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj206 = obj250 ']' 2026-02-19T13:13:54.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj206 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.679 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.679 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj207 = obj250 ']' 2026-02-19T13:13:54.679 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj207 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj208 = obj250 ']' 2026-02-19T13:13:54.698 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj208 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj209 = obj250 ']' 2026-02-19T13:13:54.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj209 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj210 = obj250 ']' 2026-02-19T13:13:54.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj210 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj211 = obj250 ']' 2026-02-19T13:13:54.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj211 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj212 = obj250 ']' 2026-02-19T13:13:54.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj212 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj213 = obj250 ']' 2026-02-19T13:13:54.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj213 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj214 = obj250 ']' 2026-02-19T13:13:54.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj214 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.824 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj215 = obj250 ']' 2026-02-19T13:13:54.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj215 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj216 = obj250 ']' 2026-02-19T13:13:54.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj216 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj217 = obj250 ']' 2026-02-19T13:13:54.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj217 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj218 = obj250 ']' 2026-02-19T13:13:54.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj218 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.897 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj219 = obj250 ']' 2026-02-19T13:13:54.897 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj219 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.914 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj220 = obj250 ']' 2026-02-19T13:13:54.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj220 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.932 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj221 = obj250 ']' 2026-02-19T13:13:54.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj221 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.949 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj222 = obj250 ']' 2026-02-19T13:13:54.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj222 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj223 = obj250 ']' 2026-02-19T13:13:54.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj223 td/test-erasure-eio/CHECK 2026-02-19T13:13:54.984 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:54.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:54.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj224 = obj250 ']' 2026-02-19T13:13:54.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj224 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj225 = obj250 ']' 2026-02-19T13:13:55.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj225 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj226 = obj250 ']' 2026-02-19T13:13:55.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj226 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj227 = obj250 ']' 2026-02-19T13:13:55.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj227 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj228 = obj250 ']' 2026-02-19T13:13:55.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj228 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj229 = obj250 ']' 2026-02-19T13:13:55.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj229 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj230 = obj250 ']' 2026-02-19T13:13:55.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj230 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj231 = obj250 ']' 2026-02-19T13:13:55.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj231 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj232 = obj250 ']' 2026-02-19T13:13:55.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj232 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj233 = obj250 ']' 2026-02-19T13:13:55.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj233 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj234 = obj250 ']' 2026-02-19T13:13:55.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj234 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj235 = obj250 ']' 2026-02-19T13:13:55.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj235 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj236 = obj250 ']' 2026-02-19T13:13:55.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj236 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj237 = obj250 ']' 2026-02-19T13:13:55.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj237 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj238 = obj250 ']' 2026-02-19T13:13:55.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj238 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj239 = obj250 ']' 2026-02-19T13:13:55.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj239 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj240 = obj250 ']' 2026-02-19T13:13:55.271 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj240 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj241 = obj250 ']' 2026-02-19T13:13:55.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj241 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj242 = obj250 ']' 2026-02-19T13:13:55.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj242 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj243 = obj250 ']' 2026-02-19T13:13:55.323 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj243 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj244 = obj250 ']' 2026-02-19T13:13:55.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj244 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.358 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj245 = obj250 ']' 2026-02-19T13:13:55.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj245 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj246 = obj250 ']' 2026-02-19T13:13:55.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj246 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj247 = obj250 ']' 2026-02-19T13:13:55.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj247 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj248 = obj250 ']' 2026-02-19T13:13:55.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj248 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj249 = obj250 ']' 2026-02-19T13:13:55.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj249 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj250 = obj250 ']' 2026-02-19T13:13:55.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:615: TEST_ec_backfill_unfound: rados -p pool-jerasure get obj250 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.468 INFO:tasks.workunit.client.0.trial111.stderr:error getting pool-jerasure/obj250: (2) No such file or directory 2026-02-19T13:13:55.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj251 = obj250 ']' 2026-02-19T13:13:55.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj251 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj252 = obj250 ']' 2026-02-19T13:13:55.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj252 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj253 = obj250 ']' 2026-02-19T13:13:55.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj253 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj254 = obj250 ']' 2026-02-19T13:13:55.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj254 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj255 = obj250 ']' 2026-02-19T13:13:55.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj255 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj256 = obj250 ']' 2026-02-19T13:13:55.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj256 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj257 = obj250 ']' 2026-02-19T13:13:55.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj257 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj258 = obj250 ']' 2026-02-19T13:13:55.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj258 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj259 = obj250 ']' 2026-02-19T13:13:55.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj259 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj260 = obj250 ']' 2026-02-19T13:13:55.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj260 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj261 = obj250 ']' 2026-02-19T13:13:55.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj261 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj262 = obj250 ']' 2026-02-19T13:13:55.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj262 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj263 = obj250 ']' 2026-02-19T13:13:55.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj263 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj264 = obj250 ']' 2026-02-19T13:13:55.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj264 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.719 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.719 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj265 = obj250 ']' 2026-02-19T13:13:55.719 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj265 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj266 = obj250 ']' 2026-02-19T13:13:55.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj266 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj267 = obj250 ']' 2026-02-19T13:13:55.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj267 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj268 = obj250 ']' 2026-02-19T13:13:55.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj268 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj269 = obj250 ']' 2026-02-19T13:13:55.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj269 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj270 = obj250 ']' 2026-02-19T13:13:55.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj270 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj271 = obj250 ']' 2026-02-19T13:13:55.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj271 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj272 = obj250 ']' 2026-02-19T13:13:55.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj272 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj273 = obj250 ']' 2026-02-19T13:13:55.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj273 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj274 = obj250 ']' 2026-02-19T13:13:55.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj274 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.898 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.899 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.899 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj275 = obj250 ']' 2026-02-19T13:13:55.899 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj275 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj276 = obj250 ']' 2026-02-19T13:13:55.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj276 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj277 = obj250 ']' 2026-02-19T13:13:55.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj277 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj278 = obj250 ']' 2026-02-19T13:13:55.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj278 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj279 = obj250 ']' 2026-02-19T13:13:55.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj279 td/test-erasure-eio/CHECK 2026-02-19T13:13:55.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:55.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:55.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj280 = obj250 ']' 2026-02-19T13:13:55.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj280 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj281 = obj250 ']' 2026-02-19T13:13:56.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj281 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.023 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj282 = obj250 ']' 2026-02-19T13:13:56.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj282 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.041 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.042 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.042 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj283 = obj250 ']' 2026-02-19T13:13:56.042 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj283 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj284 = obj250 ']' 2026-02-19T13:13:56.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj284 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj285 = obj250 ']' 2026-02-19T13:13:56.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj285 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj286 = obj250 ']' 2026-02-19T13:13:56.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj286 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.114 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj287 = obj250 ']' 2026-02-19T13:13:56.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj287 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.133 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.134 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj288 = obj250 ']' 2026-02-19T13:13:56.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj288 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj289 = obj250 ']' 2026-02-19T13:13:56.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj289 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj290 = obj250 ']' 2026-02-19T13:13:56.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj290 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj291 = obj250 ']' 2026-02-19T13:13:56.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj291 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.208 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.208 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj292 = obj250 ']' 2026-02-19T13:13:56.209 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj292 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj293 = obj250 ']' 2026-02-19T13:13:56.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj293 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj294 = obj250 ']' 2026-02-19T13:13:56.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj294 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj295 = obj250 ']' 2026-02-19T13:13:56.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj295 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj296 = obj250 ']' 2026-02-19T13:13:56.283 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj296 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj297 = obj250 ']' 2026-02-19T13:13:56.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj297 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj298 = obj250 ']' 2026-02-19T13:13:56.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj298 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj299 = obj250 ']' 2026-02-19T13:13:56.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj299 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:611: TEST_ec_backfill_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:13:56.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:613: TEST_ec_backfill_unfound: '[' obj300 = obj250 ']' 2026-02-19T13:13:56.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:617: TEST_ec_backfill_unfound: rados --pool pool-jerasure get obj300 td/test-erasure-eio/CHECK 2026-02-19T13:13:56.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:618: TEST_ec_backfill_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:622: TEST_ec_backfill_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:13:56.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:624: TEST_ec_backfill_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:13:56.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:13:56.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:13:56.637 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:13:56.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:13:56.916 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:13:56.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:13:56.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:13:56.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:13:56.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:13:56.927 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:13:56.927 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:13:56.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:13:56.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:13:56.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:13:57.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:13:57.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:13:57.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:13:57.036 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:13:57.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:13:57.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:13:57.037 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:13:57.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:13:57.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:13:57.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:13:57.039 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:13:57.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:13:57.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:13:57.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:13:57.077 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:13:57.077 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:57.077 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:57.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:13:57.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:13:57.080 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:13:57.081 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:13:57.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:13:57.082 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:13:57.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:13:57.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:13:57.083 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:13:57.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:13:57.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:13:57.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:13:57.084 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:13:57.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:13:57.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:13:57.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:13:57.087 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:13:57.087 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:57.087 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:57.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:13:57.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:13:57.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:13:57.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:13:57.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:13:57.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:57.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:57.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:13:57.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:13:57.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:13:57.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:13:57.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:13:57.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:13:57.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:13:57.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:13:57.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:13:57.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:13:57.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:13:57.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:13:57.112 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:13:57.113 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:13:57.113 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:13:57.113 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:13:57.113 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:57.113 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:57.113 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:13:57.113 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:13:57.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:13:57.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:13:57.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:13:57.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:13:57.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:13:57.138 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:13:57.203 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:13:57.203 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:13:57.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:13:57.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:13:57.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:13:57.204 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:13:57.204 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:13:57.204 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:13:57.204 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:13:57.205 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:57.205 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:57.205 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:13:57.205 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:13:57.205 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:13:57.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:13:57.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:13:57.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:13:57.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:13:57.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:13:57.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:13:57.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:13:57.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:13:57.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:13:57.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:13:57.375 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:13:57.376 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:57.376 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:57.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:13:57.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:13:57.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:13:57.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:13:57.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:13:57.493 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:13:57.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:13:58.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:13:58.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:13:58.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:13:58.507 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:13:58.507 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:58.507 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:58.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:13:58.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:13:58.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:13:58.560 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:13:58.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:13:58.568 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:13:57.130+0000 7f5994039dc0 0 load: jerasure load: lrc 2026-02-19T13:13:58.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_object_attr_read_error td/test-erasure-eio 2026-02-19T13:13:58.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:379: TEST_ec_object_attr_read_error: local dir=td/test-erasure-eio 2026-02-19T13:13:58.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:380: TEST_ec_object_attr_read_error: local objname=myobject 2026-02-19T13:13:58.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:382: TEST_ec_object_attr_read_error: setup_osds 7 2026-02-19T13:13:58.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-19T13:13:58.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:13:58.569 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-19T13:13:58.573 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-19T13:13:58.573 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:13:58.573 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:13:58.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:13:58.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:13:58.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d5d95d09-f526-40fc-9c79-c31813f26785 2026-02-19T13:13:58.577 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 d5d95d09-f526-40fc-9c79-c31813f26785 2026-02-19T13:13:58.577 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 d5d95d09-f526-40fc-9c79-c31813f26785' 2026-02-19T13:13:58.577 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:13:58.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCWDJdpyS7jIhAAD2HXT8oUy1FDQuooZ19RWA== 2026-02-19T13:13:58.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCWDJdpyS7jIhAAD2HXT8oUy1FDQuooZ19RWA=="}' 2026-02-19T13:13:58.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d5d95d09-f526-40fc-9c79-c31813f26785 -i td/test-erasure-eio/0/new.json 2026-02-19T13:13:58.803 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:13:58.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:13:58.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCWDJdpyS7jIhAAD2HXT8oUy1FDQuooZ19RWA== --osd-uuid d5d95d09-f526-40fc-9c79-c31813f26785 2026-02-19T13:13:58.830 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:58.830+0000 7f8f914fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:58.831 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:58.832+0000 7f8f914fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:58.832 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:58.832+0000 7f8f914fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:58.833 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:58.833+0000 7f8f914fe900 -1 bdev(0x55eff5096c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:13:58.833 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:58.833+0000 7f8f914fe900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:13:59.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:13:59.223 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:13:59.223 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:13:59.223 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:13:59.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:13:59.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:13:59.522 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:13:59.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:13:59.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:13:59.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:13:59.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:13:59.535 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:59.535+0000 7f3e44ccd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:59.537 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:59.537+0000 7f3e44ccd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:59.537 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:59.537+0000 7f3e44ccd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:13:59.742 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:13:59.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:13:59.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:13:59.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:13:59.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:13:59.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:13:59.743 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:13:59.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:13:59.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:13:59.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:13:59.746 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:13:59.746+0000 7f3e44ccd900 -1 Falling back to public interface 2026-02-19T13:13:59.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:00.101 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:00.101+0000 7f3e44ccd900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:14:00.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:00.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:00.964 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:00.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:00.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:00.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:01.186 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/67516159,v1:127.0.0.1:6803/67516159] [v2:127.0.0.1:6804/67516159,v1:127.0.0.1:6805/67516159] exists,up d5d95d09-f526-40fc-9c79-c31813f26785 2026-02-19T13:14:01.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:01.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:01.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:01.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:01.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=01fef127-5fe9-419f-b222-c3626edea68e 2026-02-19T13:14:01.195 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 01fef127-5fe9-419f-b222-c3626edea68e 2026-02-19T13:14:01.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 01fef127-5fe9-419f-b222-c3626edea68e' 2026-02-19T13:14:01.195 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:01.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCZDJdpyBQHDBAArG5/kAumboJ5bX+jsZme+g== 2026-02-19T13:14:01.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCZDJdpyBQHDBAArG5/kAumboJ5bX+jsZme+g=="}' 2026-02-19T13:14:01.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 01fef127-5fe9-419f-b222-c3626edea68e -i td/test-erasure-eio/1/new.json 2026-02-19T13:14:01.418 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:01.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:14:01.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCZDJdpyBQHDBAArG5/kAumboJ5bX+jsZme+g== --osd-uuid 01fef127-5fe9-419f-b222-c3626edea68e 2026-02-19T13:14:01.445 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:01.446+0000 7f0d0a8eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:01.447 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:01.447+0000 7f0d0a8eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:01.447 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:01.448+0000 7f0d0a8eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:01.448 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:01.448+0000 7f0d0a8eb900 -1 bdev(0x56510765cc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:01.448 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:01.448+0000 7f0d0a8eb900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:14:01.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:14:01.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:01.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:14:01.866 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:14:01.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:02.166 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:14:02.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:14:02.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:02.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:02.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:02.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:02.180 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:02.180+0000 7fc43887f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:02.181 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:02.182+0000 7fc43887f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:02.182 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:02.183+0000 7fc43887f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:02.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:14:02.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:02.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:14:02.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:02.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:02.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:02.388 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:02.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:02.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:02.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:02.396 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:02.397+0000 7fc43887f900 -1 Falling back to public interface 2026-02-19T13:14:02.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:02.774 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:02.775+0000 7fc43887f900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:14:03.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:03.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:03.608 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:03.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:03.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:03.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:03.830 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/4158041318,v1:127.0.0.1:6811/4158041318] [v2:127.0.0.1:6812/4158041318,v1:127.0.0.1:6813/4158041318] exists,up 01fef127-5fe9-419f-b222-c3626edea68e 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:03.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:14:03.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:03.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4cc32dff-5cac-4115-81ff-bc5a72780232 2026-02-19T13:14:03.834 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 4cc32dff-5cac-4115-81ff-bc5a72780232 2026-02-19T13:14:03.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4cc32dff-5cac-4115-81ff-bc5a72780232' 2026-02-19T13:14:03.835 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:03.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCbDJdpeQlxMhAAH7poQUn4uvE5Q7fP1M0CNA== 2026-02-19T13:14:03.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCbDJdpeQlxMhAAH7poQUn4uvE5Q7fP1M0CNA=="}' 2026-02-19T13:14:03.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4cc32dff-5cac-4115-81ff-bc5a72780232 -i td/test-erasure-eio/2/new.json 2026-02-19T13:14:04.066 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:14:04.077 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:14:04.078 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCbDJdpeQlxMhAAH7poQUn4uvE5Q7fP1M0CNA== --osd-uuid 4cc32dff-5cac-4115-81ff-bc5a72780232 2026-02-19T13:14:04.093 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:04.095+0000 7fc89a63b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:04.095 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:04.096+0000 7fc89a63b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:04.096 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:04.097+0000 7fc89a63b900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:04.096 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:04.097+0000 7fc89a63b900 -1 bdev(0x558dfe156c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:04.096 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:04.097+0000 7fc89a63b900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:14:04.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:14:04.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:04.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:14:04.529 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:14:04.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:04.829 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:14:04.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:14:04.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:04.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:04.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:04.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:04.843 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:04.845+0000 7f7c50391900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:04.845 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:04.846+0000 7f7c50391900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:04.845 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:04.847+0000 7f7c50391900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:05.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:14:05.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:05.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:14:05.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:05.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:05.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:05.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:05.051 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:05.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:05.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:05.069 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:05.070+0000 7f7c50391900 -1 Falling back to public interface 2026-02-19T13:14:05.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:05.459 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:05.460+0000 7f7c50391900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:14:06.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:06.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:06.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:06.276 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:06.276 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:06.276 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:06.499 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3923316545,v1:127.0.0.1:6819/3923316545] [v2:127.0.0.1:6820/3923316545,v1:127.0.0.1:6821/3923316545] exists,up 4cc32dff-5cac-4115-81ff-bc5a72780232 2026-02-19T13:14:06.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:06.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:06.505 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:06.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:06.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:06.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:06.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:14:06.507 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:06.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=c406d5c0-8967-4593-91e2-28d1cdc58ccb 2026-02-19T13:14:06.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 c406d5c0-8967-4593-91e2-28d1cdc58ccb' 2026-02-19T13:14:06.507 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 c406d5c0-8967-4593-91e2-28d1cdc58ccb 2026-02-19T13:14:06.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:06.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCeDJdpA6SwHhAA/hYiJIFUlqG7RHRTjp8JdA== 2026-02-19T13:14:06.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCeDJdpA6SwHhAA/hYiJIFUlqG7RHRTjp8JdA=="}' 2026-02-19T13:14:06.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new c406d5c0-8967-4593-91e2-28d1cdc58ccb -i td/test-erasure-eio/3/new.json 2026-02-19T13:14:06.733 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:14:06.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:14:06.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCeDJdpA6SwHhAA/hYiJIFUlqG7RHRTjp8JdA== --osd-uuid c406d5c0-8967-4593-91e2-28d1cdc58ccb 2026-02-19T13:14:06.761 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:06.762+0000 7f9a9a05a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:06.762 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:06.763+0000 7f9a9a05a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:06.763 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:06.764+0000 7f9a9a05a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:06.763 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:06.764+0000 7f9a9a05a900 -1 bdev(0x563a0865ec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:06.763 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:06.764+0000 7f9a9a05a900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:14:07.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:14:07.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:07.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:14:07.172 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:14:07.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:07.472 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:14:07.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:14:07.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:07.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:07.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:07.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:07.487 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:07.488+0000 7f83422d7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:07.488 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:07.490+0000 7f83422d7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:07.489 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:07.490+0000 7f83422d7900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:07.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:14:07.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:07.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:14:07.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:07.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:07.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:07.691 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:07.692 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:07.692 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:07.692 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:07.699 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:07.700+0000 7f83422d7900 -1 Falling back to public interface 2026-02-19T13:14:07.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:08.037 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:08.038+0000 7f83422d7900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:14:08.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:08.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:08.915 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:08.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:08.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:08.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:09.139 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4151998225,v1:127.0.0.1:6827/4151998225] [v2:127.0.0.1:6828/4151998225,v1:127.0.0.1:6829/4151998225] exists,up c406d5c0-8967-4593-91e2-28d1cdc58ccb 2026-02-19T13:14:09.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:09.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:09.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:09.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:09.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T13:14:09.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:09.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:09.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:09.144 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:09.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=17662d83-e5f7-42bc-9841-661341649b05 2026-02-19T13:14:09.146 INFO:tasks.workunit.client.0.trial111.stdout:add osd4 17662d83-e5f7-42bc-9841-661341649b05 2026-02-19T13:14:09.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 17662d83-e5f7-42bc-9841-661341649b05' 2026-02-19T13:14:09.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:09.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQChDJdpPLc3CRAAV/zoTrz3mJ8TJ3VHHqnYEQ== 2026-02-19T13:14:09.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQChDJdpPLc3CRAAV/zoTrz3mJ8TJ3VHHqnYEQ=="}' 2026-02-19T13:14:09.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 17662d83-e5f7-42bc-9841-661341649b05 -i td/test-erasure-eio/4/new.json 2026-02-19T13:14:09.373 INFO:tasks.workunit.client.0.trial111.stdout:4 2026-02-19T13:14:09.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T13:14:09.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQChDJdpPLc3CRAAV/zoTrz3mJ8TJ3VHHqnYEQ== --osd-uuid 17662d83-e5f7-42bc-9841-661341649b05 2026-02-19T13:14:09.401 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:09.402+0000 7f8c13f06900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:09.402 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:09.404+0000 7f8c13f06900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:09.403 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:09.405+0000 7f8c13f06900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:09.404 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:09.405+0000 7f8c13f06900 -1 bdev(0x555a927eac00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:09.404 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:09.405+0000 7f8c13f06900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T13:14:09.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T13:14:09.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:09.816 INFO:tasks.workunit.client.0.trial111.stdout:adding osd4 key to auth repository 2026-02-19T13:14:09.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T13:14:09.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:10.117 INFO:tasks.workunit.client.0.trial111.stdout:start osd.4 2026-02-19T13:14:10.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T13:14:10.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:10.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:10.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:10.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:10.132 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:10.132+0000 7f4153b95900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:10.133 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:10.133+0000 7f4153b95900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:10.134 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:10.134+0000 7f4153b95900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:10.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T13:14:10.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:10.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T13:14:10.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:10.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:10.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:10.339 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:10.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:10.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:10.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:14:10.347 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:10.348+0000 7f4153b95900 -1 Falling back to public interface 2026-02-19T13:14:10.557 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:10.772 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:10.773+0000 7f4153b95900 -1 osd.4 0 log_to_monitors true 2026-02-19T13:14:11.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:11.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:11.560 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:11.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:11.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:11.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:14:11.786 INFO:tasks.workunit.client.0.trial111.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1825665405,v1:127.0.0.1:6835/1825665405] [v2:127.0.0.1:6836/1825665405,v1:127.0.0.1:6837/1825665405] exists,up 17662d83-e5f7-42bc-9841-661341649b05 2026-02-19T13:14:11.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:11.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:11.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:11.792 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:11.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T13:14:11.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:11.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7df2db07-8f7a-4c25-b1d3-3851bfb1562a 2026-02-19T13:14:11.794 INFO:tasks.workunit.client.0.trial111.stdout:add osd5 7df2db07-8f7a-4c25-b1d3-3851bfb1562a 2026-02-19T13:14:11.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 7df2db07-8f7a-4c25-b1d3-3851bfb1562a' 2026-02-19T13:14:11.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:11.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCjDJdp6HrILxAA2zNWmW9WFRHDSgxs4fD/9w== 2026-02-19T13:14:11.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCjDJdp6HrILxAA2zNWmW9WFRHDSgxs4fD/9w=="}' 2026-02-19T13:14:11.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7df2db07-8f7a-4c25-b1d3-3851bfb1562a -i td/test-erasure-eio/5/new.json 2026-02-19T13:14:12.023 INFO:tasks.workunit.client.0.trial111.stdout:5 2026-02-19T13:14:12.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T13:14:12.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCjDJdp6HrILxAA2zNWmW9WFRHDSgxs4fD/9w== --osd-uuid 7df2db07-8f7a-4c25-b1d3-3851bfb1562a 2026-02-19T13:14:12.052 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.052+0000 7f13ade7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.053 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.053+0000 7f13ade7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.054 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.054+0000 7f13ade7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.054 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.054+0000 7f13ade7f900 -1 bdev(0x561c905a8c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:12.055 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.055+0000 7f13ade7f900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T13:14:12.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T13:14:12.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:12.477 INFO:tasks.workunit.client.0.trial111.stdout:adding osd5 key to auth repository 2026-02-19T13:14:12.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T13:14:12.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:12.775 INFO:tasks.workunit.client.0.trial111.stdout:start osd.5 2026-02-19T13:14:12.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T13:14:12.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:12.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:12.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:12.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:12.790 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.790+0000 7f46bb3d5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.792 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.792+0000 7f46bb3d5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.792 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.793+0000 7f46bb3d5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:12.978 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:12.977+0000 7f46bb3d5900 -1 Falling back to public interface 2026-02-19T13:14:12.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T13:14:12.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:12.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T13:14:12.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:12.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:12.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:12.997 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:12.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:12.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:12.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:14:13.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:13.362 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:13.362+0000 7f46bb3d5900 -1 osd.5 0 log_to_monitors true 2026-02-19T13:14:14.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:14.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:14.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:14.219 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:14.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:14.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:14:14.437 INFO:tasks.workunit.client.0.trial111.stdout:osd.5 up in weight 1 up_from 42 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/2338004013,v1:127.0.0.1:6843/2338004013] [v2:127.0.0.1:6844/2338004013,v1:127.0.0.1:6845/2338004013] exists,up 7df2db07-8f7a-4c25-b1d3-3851bfb1562a 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:14.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:14.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2e146994-c6e3-41f1-89f9-e373835d8cb2 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 2e146994-c6e3-41f1-89f9-e373835d8cb2' 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stdout:add osd6 2e146994-c6e3-41f1-89f9-e373835d8cb2 2026-02-19T13:14:14.440 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:14.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCmDJdp2xn2GhAAW/w2CNgU4ANrMr6tzoAwFw== 2026-02-19T13:14:14.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCmDJdp2xn2GhAAW/w2CNgU4ANrMr6tzoAwFw=="}' 2026-02-19T13:14:14.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2e146994-c6e3-41f1-89f9-e373835d8cb2 -i td/test-erasure-eio/6/new.json 2026-02-19T13:14:14.672 INFO:tasks.workunit.client.0.trial111.stdout:6 2026-02-19T13:14:14.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T13:14:14.685 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCmDJdp2xn2GhAAW/w2CNgU4ANrMr6tzoAwFw== --osd-uuid 2e146994-c6e3-41f1-89f9-e373835d8cb2 2026-02-19T13:14:14.701 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:14.701+0000 7fac33c47900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:14.702 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:14.702+0000 7fac33c47900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:14.703 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:14.703+0000 7fac33c47900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:14.703 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:14.703+0000 7fac33c47900 -1 bdev(0x5635e2f1cc00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:14.703 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:14.704+0000 7fac33c47900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T13:14:15.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T13:14:15.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:15.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T13:14:15.097 INFO:tasks.workunit.client.0.trial111.stdout:adding osd6 key to auth repository 2026-02-19T13:14:15.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:15.398 INFO:tasks.workunit.client.0.trial111.stdout:start osd.6 2026-02-19T13:14:15.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T13:14:15.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:15.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:15.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:15.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:15.413 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:15.414+0000 7fc7e4d12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:15.415 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:15.415+0000 7fc7e4d12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:15.415 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:15.416+0000 7fc7e4d12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:15.596 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:15.596+0000 7fc7e4d12900 -1 Falling back to public interface 2026-02-19T13:14:15.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T13:14:15.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:15.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T13:14:15.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:15.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:15.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:15.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:15.622 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:15.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:15.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:14:15.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:15.944 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:15.944+0000 7fc7e4d12900 -1 osd.6 0 log_to_monitors true 2026-02-19T13:14:16.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:16.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:16.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:16.843 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:16.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:16.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:14:17.065 INFO:tasks.workunit.client.0.trial111.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2941873017,v1:127.0.0.1:6851/2941873017] [v2:127.0.0.1:6852/2941873017,v1:127.0.0.1:6853/2941873017] exists,up 2e146994-c6e3-41f1-89f9-e373835d8cb2 2026-02-19T13:14:17.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:17.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:17.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:17.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:14:17.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:14:17.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:14:17.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:17.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:17.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:17.066 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:14:17.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:14:17.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:14:17.125 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:14:17.133 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:14:17.135 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:13:59.749+0000 7f3e44ccd900 0 load: jerasure load: lrc 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:384: TEST_ec_object_attr_read_error: local poolname=pool-jerasure 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:385: TEST_ec_object_attr_read_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:14:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T13:14:17.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:14:17.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:14:17.777 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:14:17.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:14:18.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:14:18.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:18.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:18.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:18.790 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:18.791 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:18.791 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:14:18.791 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:18.791 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:18.792 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:18.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:18.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:18.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:18.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:18.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:18.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:19.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:19.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-19T13:14:19.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-19T13:14:19.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782' 2026-02-19T13:14:19.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:19.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:19.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-19T13:14:19.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-19T13:14:19.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574855' 2026-02-19T13:14:19.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:19.251 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:14:19.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T13:14:19.329 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T13:14:19.329 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-77309411333' 2026-02-19T13:14:19.329 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:19.329 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:14:19.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T13:14:19.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T13:14:19.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-77309411333 3-111669149702' 2026-02-19T13:14:19.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:19.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:14:19.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888068 2026-02-19T13:14:19.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888068 2026-02-19T13:14:19.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-77309411333 3-111669149702 4-146028888068' 2026-02-19T13:14:19.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:19.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:14:19.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626436 2026-02-19T13:14:19.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626436 2026-02-19T13:14:19.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-77309411333 3-111669149702 4-146028888068 5-180388626436' 2026-02-19T13:14:19.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:19.565 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:14:19.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397508 2026-02-19T13:14:19.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397508 2026-02-19T13:14:19.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574855 2-77309411333 3-111669149702 4-146028888068 5-180388626436 6-210453397508' 2026-02-19T13:14:19.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:19.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-19T13:14:19.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:19.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:19.644 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803782 2026-02-19T13:14:19.644 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:19.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803782 2026-02-19T13:14:19.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-19T13:14:19.645 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803782 2026-02-19T13:14:19.645 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:19.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803780 -lt 25769803782 2026-02-19T13:14:19.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:20.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:14:20.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:21.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-19T13:14:21.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:21.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-19T13:14:21.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:21.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:14:21.093 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574855 2026-02-19T13:14:21.094 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:21.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574855 2026-02-19T13:14:21.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-19T13:14:21.095 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574855 2026-02-19T13:14:21.095 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:14:21.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574855 -lt 55834574855 2026-02-19T13:14:21.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:21.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T13:14:21.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:21.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:14:21.319 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411333 2026-02-19T13:14:21.319 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:21.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411333 2026-02-19T13:14:21.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T13:14:21.321 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411333 2026-02-19T13:14:21.321 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:14:21.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T13:14:21.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:21.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T13:14:21.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:21.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:14:21.542 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149702 2026-02-19T13:14:21.542 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:21.544 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149702 2026-02-19T13:14:21.544 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T13:14:21.544 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149702 2026-02-19T13:14:21.544 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:14:21.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-19T13:14:21.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:21.762 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888068 2026-02-19T13:14:21.762 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:21.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:14:21.764 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888068 2026-02-19T13:14:21.764 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:21.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888068 2026-02-19T13:14:21.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888068' 2026-02-19T13:14:21.765 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 146028888068 2026-02-19T13:14:21.765 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:14:21.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888068 -lt 146028888068 2026-02-19T13:14:21.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:21.986 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626436 2026-02-19T13:14:21.986 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:21.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:14:21.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-180388626436 2026-02-19T13:14:21.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:21.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626436 2026-02-19T13:14:21.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 180388626436' 2026-02-19T13:14:21.989 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 180388626436 2026-02-19T13:14:21.989 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:14:22.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626436 -lt 180388626436 2026-02-19T13:14:22.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:22.211 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397508 2026-02-19T13:14:22.211 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:22.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:14:22.213 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397508 2026-02-19T13:14:22.213 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:22.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397508 2026-02-19T13:14:22.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397508' 2026-02-19T13:14:22.214 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 210453397508 2026-02-19T13:14:22.215 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:14:22.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397508 -lt 210453397508 2026-02-19T13:14:22.435 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:14:22.435 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:22.435 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:22.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:14:22.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:14:22.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:14:22.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:14:22.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:14:22.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:14:22.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:14:22.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:14:22.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:14:22.950 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:14:22.951 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:22.951 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:23.246 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:14:23.246 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:14:23.246 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:14:23.246 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: get_primary pool-jerasure myobject 2026-02-19T13:14:23.246 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1187: get_primary: local poolname=pool-jerasure 2026-02-19T13:14:23.246 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1188: get_primary: local objectname=myobject 2026-02-19T13:14:23.246 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1190: get_primary: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:14:23.246 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1191: get_primary: jq .acting_primary 2026-02-19T13:14:23.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:387: TEST_ec_object_attr_read_error: local primary_osd=3 2026-02-19T13:14:23.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:389: TEST_ec_object_attr_read_error: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T13:14:23.467 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:23.467 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:23.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:23.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:23.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:23.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:392: TEST_ec_object_attr_read_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:14:23.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:14:23.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:14:23.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:14:23.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T13:14:23.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:395: TEST_ec_object_attr_read_error: inject_eio ec mdata pool-jerasure myobject td/test-erasure-eio 1 2026-02-19T13:14:23.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:14:23.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:14:23.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=mdata 2026-02-19T13:14:23.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:14:23.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:14:23.871 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:14:24.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=2147483647 2026-02-19T13:14:24.091 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:14:24.091 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:14:24.091 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:14:24.091 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:14:24.091 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 2147483647 5 1 0 6 2026-02-19T13:14:24.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('2147483647' '5' '1' '0' '6') 2026-02-19T13:14:24.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:14:24.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=5 2026-02-19T13:14:24.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:14:24.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/5/type 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 5 bluestore_debug_inject_read_err true 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=5 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.5 2026-02-19T13:14:24.094 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-19T13:14:24.095 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-19T13:14:24.095 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:24.095 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:24.095 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:24.095 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.5.asok 2026-02-19T13:14:24.095 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.5.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:14:24.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:14:24.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.5 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.5 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.5 ']' 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.5.asok 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:14:24.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.5.asok injectmdataerr pool-jerasure myobject 1 2026-02-19T13:14:24.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:398: TEST_ec_object_attr_read_error: activate_osd td/test-erasure-eio 3 2026-02-19T13:14:24.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:24.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:14:24.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T13:14:24.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:14:24.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:14:24.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:24.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:24.233 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:24.233 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:24.233 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:24.233 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:24.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:24.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:24.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:24.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:24.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:24.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:24.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:24.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:24.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:14:24.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:14:24.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:14:24.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T13:14:24.235 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:14:24.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:24.236 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T13:14:24.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T13:14:24.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:14:24.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:14:24.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:14:24.244 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:24.244+0000 7f5ffec04900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:24.245 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:24.245+0000 7f5ffec04900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:24.246 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:24.246+0000 7f5ffec04900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:24.415 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:24.415+0000 7f5ffec04900 -1 Falling back to public interface 2026-02-19T13:14:24.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T13:14:24.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:24.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:14:24.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:24.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:24.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:24.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:24.454 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:24.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:24.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:24.673 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:24.775 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:24.775+0000 7f5ffec04900 -1 osd.3 56 log_to_monitors true 2026-02-19T13:14:25.674 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:25.674 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:25.675 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:25.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:25.676 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:25.676 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:25.893 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 60 up_thru 60 down_at 57 last_clean_interval [26,56) [v2:127.0.0.1:6826/3812739430,v1:127.0.0.1:6827/3812739430] [v2:127.0.0.1:6828/3812739430,v1:127.0.0.1:6829/3812739430] exists,up c406d5c0-8967-4593-91e2-28d1cdc58ccb 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:401: TEST_ec_object_attr_read_error: wait_for_clean 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:25.895 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:25.896 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:14:25.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:25.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:25.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:25.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:25.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:25.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:25.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:25.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:25.972 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:26.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:26.191 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:14:26.191 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:14:26.191 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:14:26.191 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:14:26.191 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:14:26.191 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:14:26.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:26.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:26.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:26.271 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-19T13:14:26.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-19T13:14:26.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786' 2026-02-19T13:14:26.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:26.272 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:26.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574858 2026-02-19T13:14:26.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574858 2026-02-19T13:14:26.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858' 2026-02-19T13:14:26.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:26.352 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:14:26.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-19T13:14:26.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-19T13:14:26.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411336' 2026-02-19T13:14:26.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:26.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:14:26.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=257698037763 2026-02-19T13:14:26.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 257698037763 2026-02-19T13:14:26.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411336 3-257698037763' 2026-02-19T13:14:26.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:26.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:14:26.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888071 2026-02-19T13:14:26.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888071 2026-02-19T13:14:26.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411336 3-257698037763 4-146028888071' 2026-02-19T13:14:26.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:26.586 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:14:26.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626439 2026-02-19T13:14:26.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626439 2026-02-19T13:14:26.663 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411336 3-257698037763 4-146028888071 5-180388626439' 2026-02-19T13:14:26.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:26.664 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:14:26.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397511 2026-02-19T13:14:26.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397511 2026-02-19T13:14:26.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-55834574858 2-77309411336 3-257698037763 4-146028888071 5-180388626439 6-210453397511' 2026-02-19T13:14:26.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:26.743 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-19T13:14:26.743 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:26.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:26.744 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803786 2026-02-19T13:14:26.744 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:26.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803786 2026-02-19T13:14:26.746 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803786 2026-02-19T13:14:26.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-19T13:14:26.746 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:26.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803786 2026-02-19T13:14:26.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:14:27.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:14:27.967 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:28.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-19T13:14:28.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:28.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574858 2026-02-19T13:14:28.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:28.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:14:28.195 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574858 2026-02-19T13:14:28.195 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:28.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574858 2026-02-19T13:14:28.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574858' 2026-02-19T13:14:28.196 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574858 2026-02-19T13:14:28.197 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:14:28.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574858 -lt 55834574858 2026-02-19T13:14:28.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:28.421 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-19T13:14:28.421 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:28.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:14:28.423 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411336 2026-02-19T13:14:28.423 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:28.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411336 2026-02-19T13:14:28.424 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411336 2026-02-19T13:14:28.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-19T13:14:28.424 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:14:28.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411337 -lt 77309411336 2026-02-19T13:14:28.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:28.645 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-257698037763 2026-02-19T13:14:28.645 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:28.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:14:28.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-257698037763 2026-02-19T13:14:28.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:28.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=257698037763 2026-02-19T13:14:28.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 257698037763' 2026-02-19T13:14:28.647 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 257698037763 2026-02-19T13:14:28.648 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:14:28.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 257698037763 -lt 257698037763 2026-02-19T13:14:28.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:28.871 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888071 2026-02-19T13:14:28.871 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:28.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:14:28.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888071 2026-02-19T13:14:28.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:28.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888071 2026-02-19T13:14:28.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888071' 2026-02-19T13:14:28.874 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 146028888071 2026-02-19T13:14:28.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:14:29.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888072 -lt 146028888071 2026-02-19T13:14:29.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.097 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-180388626439 2026-02-19T13:14:29.097 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:14:29.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-180388626439 2026-02-19T13:14:29.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626439 2026-02-19T13:14:29.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 180388626439' 2026-02-19T13:14:29.100 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 180388626439 2026-02-19T13:14:29.101 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:14:29.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626439 -lt 180388626439 2026-02-19T13:14:29.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:29.320 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397511 2026-02-19T13:14:29.320 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:29.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:14:29.321 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397511 2026-02-19T13:14:29.321 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:29.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397511 2026-02-19T13:14:29.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397511' 2026-02-19T13:14:29.322 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 210453397511 2026-02-19T13:14:29.323 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:14:29.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397512 -lt 210453397511 2026-02-19T13:14:29.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:14:29.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:29.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:29.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:14:29.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:14:29.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:14:29.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:14:29.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:14:29.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:14:29.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:14:29.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:14:30.074 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:14:30.074 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:14:30.075 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:14:30.075 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:14:30.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:403: TEST_ec_object_attr_read_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:14:30.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-19T13:14:30.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:14:30.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:14:30.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:405: TEST_ec_object_attr_read_error: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:14:30.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:14:30.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:14:30.657 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:14:30.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:14:30.938 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:14:30.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:14:30.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:14:30.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:14:30.951 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:14:30.952 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:30.952 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:30.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:30.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:30.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:31.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:31.060 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:14:31.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:14:31.061 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:14:31.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:14:31.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:14:31.062 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:14:31.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:31.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:14:31.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:14:31.063 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:31.064 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:14:31.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:14:31.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:14:31.079 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:14:31.079 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:31.079 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:31.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:14:31.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:14:31.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:14:31.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:14:31.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:14:31.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:14:31.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:14:31.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:14:31.083 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:14:31.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:14:31.084 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:14:31.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:14:31.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:14:31.085 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:14:31.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:31.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:14:31.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:14:31.087 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:14:31.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:14:31.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:14:31.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:14:31.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:14:31.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:31.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:31.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:14:31.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:14:31.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:14:31.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:14:31.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:14:31.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:31.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:31.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:14:31.093 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:14:31.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:14:31.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:14:31.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:14:31.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:14:31.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:14:31.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:14:31.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:14:31.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:14:31.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:14:31.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:14:31.114 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:14:31.114 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:31.114 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:31.114 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:31.114 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:31.114 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:31.114 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:31.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:14:31.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:14:31.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:14:31.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:14:31.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:14:31.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:14:31.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:14:31.140 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:14:31.140 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:14:31.141 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:14:31.141 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:31.141 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:31.142 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:31.142 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:14:31.142 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:14:31.142 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:14:31.201 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:14:31.201 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:14:31.201 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:14:31.201 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:14:31.201 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:14:31.202 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:14:31.203 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:14:31.203 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:14:31.203 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:31.203 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:31.203 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:31.203 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:14:31.203 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:14:31.203 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:14:31.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:14:31.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:14:31.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:14:31.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:14:31.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:14:31.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:14:31.263 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:14:31.374 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:14:31.374 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:31.374 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:31.374 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:31.374 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:31.375 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:31.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:31.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:14:31.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:14:31.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:14:31.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:14:31.496 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:14:31.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:14:32.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:14:32.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:14:32.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:14:32.510 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:32.510 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:32.510 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:32.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:14:32.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:14:32.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:14:32.565 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:14:32.573 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:14:32.575 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:14:31.132+0000 7f0653b0adc0 0 load: jerasure load: lrc 2026-02-19T13:14:32.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_errors td/test-erasure-eio 2026-02-19T13:14:32.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:438: TEST_ec_recovery_multiple_errors: local dir=td/test-erasure-eio 2026-02-19T13:14:32.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:439: TEST_ec_recovery_multiple_errors: local objname=myobject 2026-02-19T13:14:32.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:441: TEST_ec_recovery_multiple_errors: setup_osds 9 2026-02-19T13:14:32.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=9 2026-02-19T13:14:32.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:14:32.576 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 9 - 1 2026-02-19T13:14:32.577 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 8 2026-02-19T13:14:32.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:32.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:14:32.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:32.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:32.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:32.582 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:32.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:32.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:32.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:32.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:32.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:32.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:14:32.584 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:32.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=da60a22c-557d-42bc-aebf-15e42dd18b5b 2026-02-19T13:14:32.584 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 da60a22c-557d-42bc-aebf-15e42dd18b5b 2026-02-19T13:14:32.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 da60a22c-557d-42bc-aebf-15e42dd18b5b' 2026-02-19T13:14:32.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:32.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC4DJdpFiVXIxAAs9pgsHuaZkZphoPrK+mpvQ== 2026-02-19T13:14:32.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC4DJdpFiVXIxAAs9pgsHuaZkZphoPrK+mpvQ=="}' 2026-02-19T13:14:32.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new da60a22c-557d-42bc-aebf-15e42dd18b5b -i td/test-erasure-eio/0/new.json 2026-02-19T13:14:32.810 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:32.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:14:32.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC4DJdpFiVXIxAAs9pgsHuaZkZphoPrK+mpvQ== --osd-uuid da60a22c-557d-42bc-aebf-15e42dd18b5b 2026-02-19T13:14:32.838 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:32.838+0000 7f88a67d5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:32.839 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:32.839+0000 7f88a67d5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:32.840 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:32.840+0000 7f88a67d5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:32.840 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:32.840+0000 7f88a67d5900 -1 bdev(0x564c65ca8c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:32.840 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:32.840+0000 7f88a67d5900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:14:33.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:14:33.261 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:33.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:14:33.262 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:14:33.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:33.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:14:33.560 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:14:33.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:33.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:33.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:33.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:33.574 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:33.574+0000 7f7513746900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:33.575 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:33.575+0000 7f7513746900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:33.576 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:33.576+0000 7f7513746900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:33.704 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:33.704+0000 7f7513746900 -1 Falling back to public interface 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:34.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:34.010 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:34.010+0000 7f7513746900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:14:35.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:35.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:35.005 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:35.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:35.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:35.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:14:35.230 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3281412214,v1:127.0.0.1:6803/3281412214] [v2:127.0.0.1:6804/3281412214,v1:127.0.0.1:6805/3281412214] exists,up da60a22c-557d-42bc-aebf-15e42dd18b5b 2026-02-19T13:14:35.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:35.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:35.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:35.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:35.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:14:35.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:35.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:35.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6c0ada53-4733-49d0-9770-1e2b4daa9958 2026-02-19T13:14:35.238 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 6c0ada53-4733-49d0-9770-1e2b4daa9958 2026-02-19T13:14:35.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 6c0ada53-4733-49d0-9770-1e2b4daa9958' 2026-02-19T13:14:35.238 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:35.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC7DJdpmCqkDhAAfrG75iAeKh4X+0arZyzLbA== 2026-02-19T13:14:35.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC7DJdpmCqkDhAAfrG75iAeKh4X+0arZyzLbA=="}' 2026-02-19T13:14:35.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6c0ada53-4733-49d0-9770-1e2b4daa9958 -i td/test-erasure-eio/1/new.json 2026-02-19T13:14:35.463 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:35.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:14:35.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC7DJdpmCqkDhAAfrG75iAeKh4X+0arZyzLbA== --osd-uuid 6c0ada53-4733-49d0-9770-1e2b4daa9958 2026-02-19T13:14:35.490 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:35.490+0000 7f1491995900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:35.491 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:35.492+0000 7f1491995900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:35.492 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:35.493+0000 7f1491995900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:35.493 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:35.493+0000 7f1491995900 -1 bdev(0x55eef3da6c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:35.493 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:35.493+0000 7f1491995900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:14:35.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:14:35.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:35.876 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:14:35.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:14:35.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:36.178 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:14:36.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:14:36.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:36.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:36.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:36.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:36.195 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:36.193+0000 7f03662ab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:36.196 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:36.195+0000 7f03662ab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:36.197 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:36.196+0000 7f03662ab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:36.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:14:36.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:36.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:14:36.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:36.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:36.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:36.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:36.401 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:36.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:36.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:36.412 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:36.412+0000 7f03662ab900 -1 Falling back to public interface 2026-02-19T13:14:36.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:36.743 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:36.743+0000 7f03662ab900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:14:37.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:37.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:37.621 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:37.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:37.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:37.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:14:37.846 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1476856774,v1:127.0.0.1:6811/1476856774] [v2:127.0.0.1:6812/1476856774,v1:127.0.0.1:6813/1476856774] exists,up 6c0ada53-4733-49d0-9770-1e2b4daa9958 2026-02-19T13:14:37.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:37.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:37.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:37.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:37.852 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:37.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=4ea38deb-6aed-4eda-b2cd-ed4d77d5a1ca 2026-02-19T13:14:37.854 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 4ea38deb-6aed-4eda-b2cd-ed4d77d5a1ca 2026-02-19T13:14:37.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 4ea38deb-6aed-4eda-b2cd-ed4d77d5a1ca' 2026-02-19T13:14:37.854 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:37.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC9DJdpFnllMxAALRk6IAamBs9yNRX5bJLG6g== 2026-02-19T13:14:37.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC9DJdpFnllMxAALRk6IAamBs9yNRX5bJLG6g=="}' 2026-02-19T13:14:37.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 4ea38deb-6aed-4eda-b2cd-ed4d77d5a1ca -i td/test-erasure-eio/2/new.json 2026-02-19T13:14:38.080 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:14:38.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:14:38.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC9DJdpFnllMxAALRk6IAamBs9yNRX5bJLG6g== --osd-uuid 4ea38deb-6aed-4eda-b2cd-ed4d77d5a1ca 2026-02-19T13:14:38.108 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:38.108+0000 7f4402612900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:38.109 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:38.109+0000 7f4402612900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:38.110 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:38.110+0000 7f4402612900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:38.110 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:38.111+0000 7f4402612900 -1 bdev(0x559c7b84cc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:38.111 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:38.111+0000 7f4402612900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:14:38.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:14:38.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:38.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:14:38.537 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:14:38.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:38.837 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:14:38.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:14:38.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:38.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:38.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:38.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:38.852 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:38.852+0000 7f22ad3f5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:38.853 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:38.854+0000 7f22ad3f5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:38.854 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:38.855+0000 7f22ad3f5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:39.045 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:39.046+0000 7f22ad3f5900 -1 Falling back to public interface 2026-02-19T13:14:39.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:14:39.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:39.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:14:39.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:39.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:39.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:39.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:39.060 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:39.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:39.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:39.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:39.421 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:39.423+0000 7f22ad3f5900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:14:40.280 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:40.280 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:40.280 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:40.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:40.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:40.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:14:40.503 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2777792419,v1:127.0.0.1:6819/2777792419] [v2:127.0.0.1:6820/2777792419,v1:127.0.0.1:6821/2777792419] exists,up 4ea38deb-6aed-4eda-b2cd-ed4d77d5a1ca 2026-02-19T13:14:40.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:40.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:40.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:40.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:40.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:14:40.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:40.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:40.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:40.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:40.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e73f05cf-b40a-4315-b146-e8a5ebb59247 2026-02-19T13:14:40.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 e73f05cf-b40a-4315-b146-e8a5ebb59247' 2026-02-19T13:14:40.511 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 e73f05cf-b40a-4315-b146-e8a5ebb59247 2026-02-19T13:14:40.511 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:40.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDADJdpdHTgHhAA+m3hacGRmt/1X7dpN9KQOg== 2026-02-19T13:14:40.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDADJdpdHTgHhAA+m3hacGRmt/1X7dpN9KQOg=="}' 2026-02-19T13:14:40.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e73f05cf-b40a-4315-b146-e8a5ebb59247 -i td/test-erasure-eio/3/new.json 2026-02-19T13:14:40.738 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:14:40.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:14:40.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDADJdpdHTgHhAA+m3hacGRmt/1X7dpN9KQOg== --osd-uuid e73f05cf-b40a-4315-b146-e8a5ebb59247 2026-02-19T13:14:40.765 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:40.767+0000 7f84efd89900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:40.767 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:40.768+0000 7f84efd89900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:40.767 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:40.769+0000 7f84efd89900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:40.768 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:40.769+0000 7f84efd89900 -1 bdev(0x55bf2f8b0c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:40.768 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:40.769+0000 7f84efd89900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:14:41.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:14:41.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:41.208 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:14:41.208 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:14:41.209 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:41.508 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:14:41.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:14:41.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:41.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:41.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:41.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:41.522 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:41.524+0000 7f7704025900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:41.524 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:41.525+0000 7f7704025900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:41.525 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:41.526+0000 7f7704025900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:41.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:14:41.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:41.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:14:41.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:41.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:41.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:41.726 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:41.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:41.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:41.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:41.732 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:41.734+0000 7f7704025900 -1 Falling back to public interface 2026-02-19T13:14:41.946 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:42.152 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:42.154+0000 7f7704025900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:14:42.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:42.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:42.948 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:42.949 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:42.949 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:42.949 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:14:43.169 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3479831354,v1:127.0.0.1:6827/3479831354] [v2:127.0.0.1:6828/3479831354,v1:127.0.0.1:6829/3479831354] exists,up e73f05cf-b40a-4315-b146-e8a5ebb59247 2026-02-19T13:14:43.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:43.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:43.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:43.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:43.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T13:14:43.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:43.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:43.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:43.174 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:43.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=de52898c-9165-41f9-9312-fe7c1b0c392d 2026-02-19T13:14:43.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 de52898c-9165-41f9-9312-fe7c1b0c392d' 2026-02-19T13:14:43.177 INFO:tasks.workunit.client.0.trial111.stdout:add osd4 de52898c-9165-41f9-9312-fe7c1b0c392d 2026-02-19T13:14:43.177 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:43.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDDDJdpztkBCxAAZKPLe6nK4f/2cxJMK+IKtQ== 2026-02-19T13:14:43.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDDDJdpztkBCxAAZKPLe6nK4f/2cxJMK+IKtQ=="}' 2026-02-19T13:14:43.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new de52898c-9165-41f9-9312-fe7c1b0c392d -i td/test-erasure-eio/4/new.json 2026-02-19T13:14:43.402 INFO:tasks.workunit.client.0.trial111.stdout:4 2026-02-19T13:14:43.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T13:14:43.415 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDDDJdpztkBCxAAZKPLe6nK4f/2cxJMK+IKtQ== --osd-uuid de52898c-9165-41f9-9312-fe7c1b0c392d 2026-02-19T13:14:43.430 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:43.431+0000 7f0b113af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:43.431 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:43.433+0000 7f0b113af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:43.432 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:43.434+0000 7f0b113af900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:43.433 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:43.434+0000 7f0b113af900 -1 bdev(0x55c31a1fcc00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:43.433 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:43.434+0000 7f0b113af900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T13:14:43.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T13:14:43.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:43.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T13:14:43.840 INFO:tasks.workunit.client.0.trial111.stdout:adding osd4 key to auth repository 2026-02-19T13:14:43.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:44.146 INFO:tasks.workunit.client.0.trial111.stdout:start osd.4 2026-02-19T13:14:44.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T13:14:44.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:44.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:44.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:44.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:44.161 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:44.162+0000 7f5c1ca0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:44.162 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:44.164+0000 7f5c1ca0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:44.163 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:44.165+0000 7f5c1ca0f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:44.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:14:44.376 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:44.378+0000 7f5c1ca0f900 -1 Falling back to public interface 2026-02-19T13:14:44.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:44.764 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:44.765+0000 7f5c1ca0f900 -1 osd.4 0 log_to_monitors true 2026-02-19T13:14:45.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:45.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:45.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:45.589 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:45.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:45.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:14:45.807 INFO:tasks.workunit.client.0.trial111.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/3002052850,v1:127.0.0.1:6835/3002052850] [v2:127.0.0.1:6836/3002052850,v1:127.0.0.1:6837/3002052850] exists,up de52898c-9165-41f9-9312-fe7c1b0c392d 2026-02-19T13:14:45.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:45.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:45.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:45.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:45.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T13:14:45.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:45.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:45.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:45.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:45.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=82ee928c-7699-434c-af22-3b18c7cd2467 2026-02-19T13:14:45.815 INFO:tasks.workunit.client.0.trial111.stdout:add osd5 82ee928c-7699-434c-af22-3b18c7cd2467 2026-02-19T13:14:45.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 82ee928c-7699-434c-af22-3b18c7cd2467' 2026-02-19T13:14:45.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:45.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDFDJdpU60GMRAAT47ACZbr0PjRFtS/8Kkv8g== 2026-02-19T13:14:45.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDFDJdpU60GMRAAT47ACZbr0PjRFtS/8Kkv8g=="}' 2026-02-19T13:14:45.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 82ee928c-7699-434c-af22-3b18c7cd2467 -i td/test-erasure-eio/5/new.json 2026-02-19T13:14:46.040 INFO:tasks.workunit.client.0.trial111.stdout:5 2026-02-19T13:14:46.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T13:14:46.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDFDJdpU60GMRAAT47ACZbr0PjRFtS/8Kkv8g== --osd-uuid 82ee928c-7699-434c-af22-3b18c7cd2467 2026-02-19T13:14:46.067 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.068+0000 7efe3a341900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.068 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.070+0000 7efe3a341900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.069 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.071+0000 7efe3a341900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.069 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.071+0000 7efe3a341900 -1 bdev(0x5580ab702c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:46.070 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.071+0000 7efe3a341900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T13:14:46.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T13:14:46.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:46.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T13:14:46.479 INFO:tasks.workunit.client.0.trial111.stdout:adding osd5 key to auth repository 2026-02-19T13:14:46.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:46.776 INFO:tasks.workunit.client.0.trial111.stdout:start osd.5 2026-02-19T13:14:46.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T13:14:46.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:46.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:46.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:46.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:46.791 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.791+0000 7fef688e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.792 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.793+0000 7fef688e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.793 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.793+0000 7fef688e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:46.984 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:46.984+0000 7fef688e4900 -1 Falling back to public interface 2026-02-19T13:14:46.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T13:14:46.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:46.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T13:14:46.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:46.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:46.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:46.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:46.998 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:46.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:46.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:14:47.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:47.362 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:47.362+0000 7fef688e4900 -1 osd.5 0 log_to_monitors true 2026-02-19T13:14:48.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:48.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:48.222 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:48.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:48.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:48.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:14:48.442 INFO:tasks.workunit.client.0.trial111.stdout:osd.5 up in weight 1 up_from 43 up_thru 44 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3465450970,v1:127.0.0.1:6843/3465450970] [v2:127.0.0.1:6844/3465450970,v1:127.0.0.1:6845/3465450970] exists,up 82ee928c-7699-434c-af22-3b18c7cd2467 2026-02-19T13:14:48.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:48.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:48.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:48.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:48.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T13:14:48.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:48.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:48.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T13:14:48.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:48.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T13:14:48.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:48.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:48.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:48.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:48.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T13:14:48.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T13:14:48.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:48.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:48.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:48.449 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:48.449 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:48.449 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:48.449 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:48.449 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:48.449 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:48.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:48.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:48.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T13:14:48.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:48.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ad65afa0-0552-42c0-81d3-0808dbe01ef7 2026-02-19T13:14:48.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 ad65afa0-0552-42c0-81d3-0808dbe01ef7' 2026-02-19T13:14:48.451 INFO:tasks.workunit.client.0.trial111.stdout:add osd6 ad65afa0-0552-42c0-81d3-0808dbe01ef7 2026-02-19T13:14:48.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:48.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDIDJdpe2tIGxAAYgYOvNdjfqwdm9NWa87FaA== 2026-02-19T13:14:48.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDIDJdpe2tIGxAAYgYOvNdjfqwdm9NWa87FaA=="}' 2026-02-19T13:14:48.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ad65afa0-0552-42c0-81d3-0808dbe01ef7 -i td/test-erasure-eio/6/new.json 2026-02-19T13:14:48.676 INFO:tasks.workunit.client.0.trial111.stdout:6 2026-02-19T13:14:48.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T13:14:48.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDIDJdpe2tIGxAAYgYOvNdjfqwdm9NWa87FaA== --osd-uuid ad65afa0-0552-42c0-81d3-0808dbe01ef7 2026-02-19T13:14:48.705 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:48.705+0000 7f752c46a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:48.706 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:48.706+0000 7f752c46a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:48.707 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:48.707+0000 7f752c46a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:48.708 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:48.708+0000 7f752c46a900 -1 bdev(0x55bb523f8c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:48.708 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:48.708+0000 7f752c46a900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T13:14:49.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T13:14:49.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:49.148 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T13:14:49.148 INFO:tasks.workunit.client.0.trial111.stdout:adding osd6 key to auth repository 2026-02-19T13:14:49.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:49.446 INFO:tasks.workunit.client.0.trial111.stdout:start osd.6 2026-02-19T13:14:49.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T13:14:49.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:49.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:49.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:49.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:49.461 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:49.461+0000 7f37e7df0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:49.462 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:49.462+0000 7f37e7df0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:49.463 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:49.463+0000 7f37e7df0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:49.663 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:49.663+0000 7f37e7df0900 -1 Falling back to public interface 2026-02-19T13:14:49.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T13:14:49.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:49.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T13:14:49.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:49.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:49.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:49.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:49.669 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:49.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:49.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:14:49.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:50.053 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:50.053+0000 7f37e7df0900 -1 osd.6 0 log_to_monitors true 2026-02-19T13:14:50.888 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:50.889 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:50.889 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:50.889 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:50.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:50.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:14:51.111 INFO:tasks.workunit.client.0.trial111.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2182919201,v1:127.0.0.1:6851/2182919201] [v2:127.0.0.1:6852/2182919201,v1:127.0.0.1:6853/2182919201] exists,up ad65afa0-0552-42c0-81d3-0808dbe01ef7 2026-02-19T13:14:51.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:51.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:51.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:51.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:51.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 7 2026-02-19T13:14:51.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:51.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:51.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=7 2026-02-19T13:14:51.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/7 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/7' 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/7/journal' 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:51.117 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:51.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/7 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:51.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=805ca1e9-dfec-480c-90bc-c04d4fae71be 2026-02-19T13:14:51.120 INFO:tasks.workunit.client.0.trial111.stdout:add osd7 805ca1e9-dfec-480c-90bc-c04d4fae71be 2026-02-19T13:14:51.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd7 805ca1e9-dfec-480c-90bc-c04d4fae71be' 2026-02-19T13:14:51.120 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:51.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDLDJdpte2NBxAAjwnNqrpdGuerqupRxMeU5w== 2026-02-19T13:14:51.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDLDJdpte2NBxAAjwnNqrpdGuerqupRxMeU5w=="}' 2026-02-19T13:14:51.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 805ca1e9-dfec-480c-90bc-c04d4fae71be -i td/test-erasure-eio/7/new.json 2026-02-19T13:14:51.346 INFO:tasks.workunit.client.0.trial111.stdout:7 2026-02-19T13:14:51.358 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/7/new.json 2026-02-19T13:14:51.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 7 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDLDJdpte2NBxAAjwnNqrpdGuerqupRxMeU5w== --osd-uuid 805ca1e9-dfec-480c-90bc-c04d4fae71be 2026-02-19T13:14:51.374 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:51.374+0000 7fee2ba35900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:51.375 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:51.376+0000 7fee2ba35900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:51.376 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:51.377+0000 7fee2ba35900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:51.377 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:51.377+0000 7fee2ba35900 -1 bdev(0x5611af4cac00 td/test-erasure-eio/7/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:51.377 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:51.377+0000 7fee2ba35900 -1 bluestore(td/test-erasure-eio/7) _read_fsid unparsable uuid 2026-02-19T13:14:51.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/7/keyring 2026-02-19T13:14:51.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:51.820 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd7 key to auth repository 2026-02-19T13:14:51.820 INFO:tasks.workunit.client.0.trial111.stdout:adding osd7 key to auth repository 2026-02-19T13:14:51.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/7/keyring auth add osd.7 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:52.128 INFO:tasks.workunit.client.0.trial111.stdout:start osd.7 2026-02-19T13:14:52.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.7 2026-02-19T13:14:52.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 7 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/7 --osd-journal=td/test-erasure-eio/7/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:52.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:52.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:52.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:52.143 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:52.143+0000 7f14e89a9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:52.144 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:52.144+0000 7f14e89a9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:52.145 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:52.145+0000 7f14e89a9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:52.349 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 7 2026-02-19T13:14:52.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:52.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=7 2026-02-19T13:14:52.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:52.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:52.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:52.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:52.350 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:52.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:52.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T13:14:52.372 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:52.372+0000 7f14e89a9900 -1 Falling back to public interface 2026-02-19T13:14:52.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:52.738 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:52.738+0000 7f14e89a9900 -1 osd.7 0 log_to_monitors true 2026-02-19T13:14:53.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:53.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:53.571 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:53.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:53.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:53.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.7 up' 2026-02-19T13:14:53.792 INFO:tasks.workunit.client.0.trial111.stdout:osd.7 up in weight 1 up_from 56 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6858/1267239416,v1:127.0.0.1:6859/1267239416] [v2:127.0.0.1:6860/1267239416,v1:127.0.0.1:6861/1267239416] exists,up 805ca1e9-dfec-480c-90bc-c04d4fae71be 2026-02-19T13:14:53.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:53.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:53.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:53.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 8 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=8 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/8 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:14:53.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/8' 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/8/journal' 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:14:53.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/8 2026-02-19T13:14:53.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:14:53.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f8c1be0a-b6b7-408d-9fdd-531c5c3b67d2 2026-02-19T13:14:53.800 INFO:tasks.workunit.client.0.trial111.stdout:add osd8 f8c1be0a-b6b7-408d-9fdd-531c5c3b67d2 2026-02-19T13:14:53.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd8 f8c1be0a-b6b7-408d-9fdd-531c5c3b67d2' 2026-02-19T13:14:53.801 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:14:53.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDNDJdpOigtMBAAnw8fmF+e4VIb1GMoV9yfbQ== 2026-02-19T13:14:53.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDNDJdpOigtMBAAnw8fmF+e4VIb1GMoV9yfbQ=="}' 2026-02-19T13:14:53.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f8c1be0a-b6b7-408d-9fdd-531c5c3b67d2 -i td/test-erasure-eio/8/new.json 2026-02-19T13:14:54.030 INFO:tasks.workunit.client.0.trial111.stdout:8 2026-02-19T13:14:54.041 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/8/new.json 2026-02-19T13:14:54.042 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 8 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDNDJdpOigtMBAAnw8fmF+e4VIb1GMoV9yfbQ== --osd-uuid f8c1be0a-b6b7-408d-9fdd-531c5c3b67d2 2026-02-19T13:14:54.058 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:54.058+0000 7f86f0429900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:54.059 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:54.059+0000 7f86f0429900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:54.060 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:54.060+0000 7f86f0429900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:54.060 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:54.060+0000 7f86f0429900 -1 bdev(0x56297cf00c00 td/test-erasure-eio/8/block) open stat got: (1) Operation not permitted 2026-02-19T13:14:54.061 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:54.061+0000 7f86f0429900 -1 bluestore(td/test-erasure-eio/8) _read_fsid unparsable uuid 2026-02-19T13:14:54.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/8/keyring 2026-02-19T13:14:54.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:14:54.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd8 key to auth repository 2026-02-19T13:14:54.495 INFO:tasks.workunit.client.0.trial111.stdout:adding osd8 key to auth repository 2026-02-19T13:14:54.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/8/keyring auth add osd.8 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:14:54.795 INFO:tasks.workunit.client.0.trial111.stdout:start osd.8 2026-02-19T13:14:54.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.8 2026-02-19T13:14:54.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 8 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/8 --osd-journal=td/test-erasure-eio/8/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:14:54.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:14:54.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:14:54.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:14:54.810 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:54.810+0000 7f5845422900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:54.811 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:54.811+0000 7f5845422900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:54.812 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:54.812+0000 7f5845422900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 8 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=8 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:14:55.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:55.018 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:14:55.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:14:55.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:55.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T13:14:55.019 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:55.018+0000 7f5845422900 -1 Falling back to public interface 2026-02-19T13:14:55.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:14:55.373 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:14:55.373+0000 7f5845422900 -1 osd.8 0 log_to_monitors true 2026-02-19T13:14:56.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:14:56.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:14:56.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:14:56.238 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:14:56.239 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:14:56.239 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.8 up' 2026-02-19T13:14:56.458 INFO:tasks.workunit.client.0.trial111.stdout:osd.8 up in weight 1 up_from 64 up_thru 65 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6866/1971886313,v1:127.0.0.1:6867/1971886313] [v2:127.0.0.1:6868/1971886313,v1:127.0.0.1:6869/1971886313] exists,up f8c1be0a-b6b7-408d-9fdd-531c5c3b67d2 2026-02-19T13:14:56.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:14:56.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:14:56.517 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:14:56.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:14:56.526 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:14:33.708+0000 7f7513746900 0 load: jerasure load: lrc 2026-02-19T13:14:56.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:443: TEST_ec_recovery_multiple_errors: local poolname=pool-jerasure 2026-02-19T13:14:56.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:444: TEST_ec_recovery_multiple_errors: create_erasure_coded_pool pool-jerasure 4 4 2026-02-19T13:14:56.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:14:56.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:14:56.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=4 2026-02-19T13:14:56.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:14:56.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=4 2026-02-19T13:14:56.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:14:56.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=4 m=4 crush-failure-domain=osd 2026-02-19T13:14:56.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:14:56.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:14:57.167 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:14:57.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:14:58.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:14:58.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:14:58.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:14:58.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:14:58.182 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:14:58.182 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:14:58.182 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:14:58.183 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:14:58.183 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:14:58.183 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:14:58.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:14:58.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:14:58.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:14:58.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:14:58.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:14:58.260 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:14:58.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:14:58.478 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:14:58.478 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:14:58.478 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:14:58.478 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:14:58.478 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:14:58.478 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:14:58.479 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:14:58.479 INFO:tasks.workunit.client.0.trial111.stderr:8' 2026-02-19T13:14:58.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:14:58.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:58.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:14:58.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803783 2026-02-19T13:14:58.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803783 2026-02-19T13:14:58.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783' 2026-02-19T13:14:58.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:58.560 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:14:58.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-19T13:14:58.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-19T13:14:58.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607559' 2026-02-19T13:14:58.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:58.639 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:14:58.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411334 2026-02-19T13:14:58.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411334 2026-02-19T13:14:58.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334' 2026-02-19T13:14:58.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:58.718 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:14:58.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T13:14:58.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T13:14:58.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702' 2026-02-19T13:14:58.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:58.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:14:58.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855365 2026-02-19T13:14:58.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855365 2026-02-19T13:14:58.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-150323855365' 2026-02-19T13:14:58.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:58.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:14:58.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593733 2026-02-19T13:14:58.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593733 2026-02-19T13:14:58.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-150323855365 5-184683593733' 2026-02-19T13:14:58.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:58.954 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:14:59.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397508 2026-02-19T13:14:59.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397508 2026-02-19T13:14:59.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-150323855365 5-184683593733 6-210453397508' 2026-02-19T13:14:59.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:59.032 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:14:59.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=240518168580 2026-02-19T13:14:59.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 240518168580 2026-02-19T13:14:59.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-150323855365 5-184683593733 6-210453397508 7-240518168580' 2026-02-19T13:14:59.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:14:59.111 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:14:59.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=274877906947 2026-02-19T13:14:59.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 274877906947 2026-02-19T13:14:59.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607559 2-77309411334 3-111669149702 4-150323855365 5-184683593733 6-210453397508 7-240518168580 8-274877906947' 2026-02-19T13:14:59.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:14:59.191 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803783 2026-02-19T13:14:59.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:14:59.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:14:59.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803783 2026-02-19T13:14:59.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:14:59.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803783 2026-02-19T13:14:59.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-19T13:14:59.194 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803783 2026-02-19T13:14:59.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:14:59.415 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803783 2026-02-19T13:14:59.415 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:00.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:00.416 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:00.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803783 2026-02-19T13:15:00.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:00.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-19T13:15:00.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:00.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:15:00.643 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607559 2026-02-19T13:15:00.644 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:00.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607559 2026-02-19T13:15:00.645 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 51539607559 2026-02-19T13:15:00.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-19T13:15:00.645 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:15:00.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-19T13:15:00.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:00.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411334 2026-02-19T13:15:00.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:00.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:15:00.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411334 2026-02-19T13:15:00.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:00.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411334 2026-02-19T13:15:00.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411334' 2026-02-19T13:15:00.869 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411334 2026-02-19T13:15:00.869 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:15:01.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411334 -lt 77309411334 2026-02-19T13:15:01.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:01.093 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T13:15:01.093 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:01.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:15:01.094 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149702 2026-02-19T13:15:01.095 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:01.095 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149702 2026-02-19T13:15:01.096 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149702 2026-02-19T13:15:01.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T13:15:01.096 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:15:01.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-19T13:15:01.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:01.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855365 2026-02-19T13:15:01.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:01.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:15:01.319 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855365 2026-02-19T13:15:01.319 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:01.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855365 2026-02-19T13:15:01.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855365' 2026-02-19T13:15:01.321 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 150323855365 2026-02-19T13:15:01.321 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:15:01.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-19T13:15:01.543 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:01.543 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593733 2026-02-19T13:15:01.543 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:01.544 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:15:01.544 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-184683593733 2026-02-19T13:15:01.544 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593733 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 184683593733' 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 184683593733 2026-02-19T13:15:01.546 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:15:01.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593733 -lt 184683593733 2026-02-19T13:15:01.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:01.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397508 2026-02-19T13:15:01.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:01.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:15:01.769 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397508 2026-02-19T13:15:01.769 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:01.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397508 2026-02-19T13:15:01.771 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 210453397508 2026-02-19T13:15:01.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397508' 2026-02-19T13:15:01.771 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:15:01.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397509 -lt 210453397508 2026-02-19T13:15:01.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:01.992 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 7-240518168580 2026-02-19T13:15:01.993 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:01.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=7 2026-02-19T13:15:01.994 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 7-240518168580 2026-02-19T13:15:01.994 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:01.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=240518168580 2026-02-19T13:15:01.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.7 seq 240518168580' 2026-02-19T13:15:01.995 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.7 seq 240518168580 2026-02-19T13:15:01.995 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 7 2026-02-19T13:15:02.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 240518168580 -lt 240518168580 2026-02-19T13:15:02.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:02.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-274877906947 2026-02-19T13:15:02.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:02.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:15:02.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-274877906947 2026-02-19T13:15:02.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:02.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=274877906947 2026-02-19T13:15:02.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 274877906947' 2026-02-19T13:15:02.220 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.8 seq 274877906947 2026-02-19T13:15:02.221 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:15:02.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 274877906948 -lt 274877906947 2026-02-19T13:15:02.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:02.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:02.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:02.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:15:02.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:02.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:02.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:02.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:02.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:02.737 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:02.737 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:02.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:15:02.957 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:02.957 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:02.957 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:03.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:15:03.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:446: TEST_ec_recovery_multiple_errors: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:15:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:15:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T13:15:03.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:447: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-19T13:15:03.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:15:03.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:15:03.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:15:03.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:15:03.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:15:03.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:15:03.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:15:03.289 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:15:03.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:15:03.512 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:7' 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-19T13:15:03.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:15:03.514 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T13:15:03.515 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:15:03.515 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:03.516 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:15:03.517 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:15:03.580 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:15:03.580 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:15:03.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:15:03.581 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-19T13:15:03.582 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:15:03.582 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:15:03.582 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:03.582 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:03.582 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:03.582 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:15:03.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:15:03.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-19T13:15:03.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:450: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 3 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:15:03.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:15:03.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=3 2026-02-19T13:15:03.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:15:03.647 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-19T13:15:03.647 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:15:03.647 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:15:03.647 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:15:03.647 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:15:03.869 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:15:03.869 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:15:03.869 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:15:03.869 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:15:03.869 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr:7' 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=0 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:15:03.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/0/type 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 0 bluestore_debug_inject_read_err true 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=0 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.0 2026-02-19T13:15:03.873 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:15:03.874 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:15:03.874 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:03.874 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:03.874 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:03.874 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:15:03.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.0.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:15:03.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:15:03.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:15:03.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:15:03.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.0 2026-02-19T13:15:03.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:15:03.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:15:03.939 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:03.939 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:03.939 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:03.939 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:15:03.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:15:03.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok injectdataerr pool-jerasure myobject 3 2026-02-19T13:15:04.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:451: TEST_ec_recovery_multiple_errors: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 4 2026-02-19T13:15:04.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:15:04.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:15:04.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:15:04.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:15:04.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:15:04.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=4 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:15:04.005 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:15:04.225 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:15:04.225 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:15:04.225 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:15:04.225 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:15:04.225 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr:7' 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=6 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:15:04.226 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/6/type 2026-02-19T13:15:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:15:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 6 bluestore_debug_inject_read_err true 2026-02-19T13:15:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:15:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=6 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.6 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.6.asok 2026-02-19T13:15:04.229 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.6.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:15:04.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:15:04.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:15:04.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:15:04.296 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.6 2026-02-19T13:15:04.296 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.6 2026-02-19T13:15:04.296 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.6 ']' 2026-02-19T13:15:04.296 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:04.296 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:04.296 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:04.296 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.6.asok 2026-02-19T13:15:04.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:15:04.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.6.asok injectdataerr pool-jerasure myobject 4 2026-02-19T13:15:04.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: get_osds pool-jerasure myobject 2026-02-19T13:15:04.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:15:04.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:15:04.362 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:15:04.362 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:15:04.584 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:15:04.584 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:15:04.584 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:15:04.584 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:15:04.584 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:15:04.584 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:15:04.584 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr:7' 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2 4 7 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: initial_osds=('3' '5' '1' '0' '6' '2' '4' '7') 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:453: TEST_ec_recovery_multiple_errors: local -a initial_osds 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:454: TEST_ec_recovery_multiple_errors: local last_osd=7 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:456: TEST_ec_recovery_multiple_errors: kill_daemons td/test-erasure-eio TERM osd.7 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:04.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:04.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:04.891 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:457: TEST_ec_recovery_multiple_errors: ceph osd down 7 2026-02-19T13:15:05.108 INFO:tasks.workunit.client.0.trial111.stderr:osd.7 is already down. 2026-02-19T13:15:05.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:458: TEST_ec_recovery_multiple_errors: ceph osd out 7 2026-02-19T13:15:05.362 INFO:tasks.workunit.client.0.trial111.stderr:osd.7 is already out. 2026-02-19T13:15:05.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:461: TEST_ec_recovery_multiple_errors: wait_for_clean 2026-02-19T13:15:05.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:15:05.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:15:05.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:15:05.374 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:15:05.375 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:05.375 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:15:05.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:05.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:05.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:05.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:15:05.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:15:05.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:15:05.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:15:05.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:05.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:05.673 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:15:05.673 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:6 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:7 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:8' 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:05.674 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:05.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-19T13:15:05.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-19T13:15:05.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787' 2026-02-19T13:15:05.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:05.752 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:15:05.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607562 2026-02-19T13:15:05.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607562 2026-02-19T13:15:05.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-51539607562' 2026-02-19T13:15:05.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:05.830 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:15:05.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411338 2026-02-19T13:15:05.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411338 2026-02-19T13:15:05.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338' 2026-02-19T13:15:05.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:05.911 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:15:05.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149705 2026-02-19T13:15:05.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149705 2026-02-19T13:15:05.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-111669149705' 2026-02-19T13:15:05.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:05.990 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:15:06.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855369 2026-02-19T13:15:06.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855369 2026-02-19T13:15:06.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-111669149705 4-150323855369' 2026-02-19T13:15:06.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:06.073 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:15:06.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593736 2026-02-19T13:15:06.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593736 2026-02-19T13:15:06.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-111669149705 4-150323855369 5-184683593736' 2026-02-19T13:15:06.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:06.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:15:06.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397512 2026-02-19T13:15:06.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397512 2026-02-19T13:15:06.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-111669149705 4-150323855369 5-184683593736 6-210453397512' 2026-02-19T13:15:06.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:06.231 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.7 flush_pg_stats 2026-02-19T13:15:06.291 INFO:tasks.workunit.client.0.trial111.stderr:Error ENXIO: problem getting command descriptions from osd.7 2026-02-19T13:15:06.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-19T13:15:06.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-19T13:15:06.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-19T13:15:06.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:06.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.8 flush_pg_stats 2026-02-19T13:15:06.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=274877906950 2026-02-19T13:15:06.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 274877906950 2026-02-19T13:15:06.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-51539607562 2-77309411338 3-111669149705 4-150323855369 5-184683593736 6-210453397512 8-274877906950' 2026-02-19T13:15:06.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:06.374 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-19T13:15:06.374 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:06.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:06.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803787 2026-02-19T13:15:06.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:06.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803787 2026-02-19T13:15:06.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-19T13:15:06.376 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803787 2026-02-19T13:15:06.377 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:06.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-19T13:15:06.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:06.599 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607562 2026-02-19T13:15:06.599 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:06.600 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:15:06.601 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607562 2026-02-19T13:15:06.601 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:06.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607562 2026-02-19T13:15:06.602 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 51539607562 2026-02-19T13:15:06.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607562' 2026-02-19T13:15:06.602 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:15:06.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607562 -lt 51539607562 2026-02-19T13:15:06.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:06.822 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411338 2026-02-19T13:15:06.823 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:06.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:15:06.824 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411338 2026-02-19T13:15:06.824 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:06.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411338 2026-02-19T13:15:06.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411338' 2026-02-19T13:15:06.825 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411338 2026-02-19T13:15:06.826 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:15:07.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411336 -lt 77309411338 2026-02-19T13:15:07.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:08.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:08.046 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:15:08.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411338 -lt 77309411338 2026-02-19T13:15:08.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:08.269 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149705 2026-02-19T13:15:08.269 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:08.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:15:08.271 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149705 2026-02-19T13:15:08.271 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:08.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149705 2026-02-19T13:15:08.272 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149705 2026-02-19T13:15:08.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-19T13:15:08.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:15:08.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-19T13:15:08.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:08.495 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855369 2026-02-19T13:15:08.495 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:08.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:15:08.496 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855369 2026-02-19T13:15:08.496 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:08.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855369 2026-02-19T13:15:08.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855369' 2026-02-19T13:15:08.498 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 150323855369 2026-02-19T13:15:08.498 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:15:08.719 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855369 -lt 150323855369 2026-02-19T13:15:08.719 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:08.720 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593736 2026-02-19T13:15:08.720 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:08.720 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:15:08.721 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-184683593736 2026-02-19T13:15:08.721 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:08.722 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593736 2026-02-19T13:15:08.722 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 184683593736 2026-02-19T13:15:08.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 184683593736' 2026-02-19T13:15:08.723 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:15:08.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593736 -lt 184683593736 2026-02-19T13:15:08.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:08.942 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397512 2026-02-19T13:15:08.942 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:08.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:15:08.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397512 2026-02-19T13:15:08.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:08.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397512 2026-02-19T13:15:08.945 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 210453397512 2026-02-19T13:15:08.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397512' 2026-02-19T13:15:08.945 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:15:09.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397512 -lt 210453397512 2026-02-19T13:15:09.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:09.169 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 8-274877906950 2026-02-19T13:15:09.169 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:09.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=8 2026-02-19T13:15:09.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 8-274877906950 2026-02-19T13:15:09.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:09.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=274877906950 2026-02-19T13:15:09.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.8 seq 274877906950' 2026-02-19T13:15:09.171 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.8 seq 274877906950 2026-02-19T13:15:09.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 8 2026-02-19T13:15:09.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 274877906951 -lt 274877906950 2026-02-19T13:15:09.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:09.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:09.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:09.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:15:09.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:09.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:09.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:09.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:09.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:09.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:09.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:09.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:15:09.912 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:09.912 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:09.912 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:10.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:15:10.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:10.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:10.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:463: TEST_ec_recovery_multiple_errors: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-19T13:15:10.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:15:10.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:15:10.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-19T13:15:10.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:15:10.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:15:10.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-19T13:15:10.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:15:10.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:15:10.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:465: TEST_ec_recovery_multiple_errors: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:15:10.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:15:10.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:15:10.498 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:15:10.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:15:10.779 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:10.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:10.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:10.900 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:10.901 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:10.901 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:10.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:10.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:10.902 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:10.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:10.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:10.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:10.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:10.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:10.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:15:10.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:15:10.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:10.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:10.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:10.924 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:15:10.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:10.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:10.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:10.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:10.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:10.928 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:10.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:10.929 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:10.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:10.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:10.930 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:10.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:10.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:10.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:10.931 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:10.932 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:10.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:15:10.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:15:10.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:10.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:10.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:10.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:15:10.935 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:10.935 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:10.935 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:15:10.937 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:15:10.937 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:10.937 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:10.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:15:10.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:15:10.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:15:10.958 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:15:10.958 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:10.958 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:10.958 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:10.959 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:10.959 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:10.959 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:10.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:15:10.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:15:10.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:15:10.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:15:10.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:15:10.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:15:10.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:15:11.044 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:15:11.045 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:15:11.045 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:15:11.045 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:15:11.045 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:15:11.045 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:15:11.045 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:11.045 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:11.046 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:11.046 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:11.046 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:11.046 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:15:11.046 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:15:11.047 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:15:11.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:15:11.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:15:11.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:15:11.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:15:11.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:15:11.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:15:11.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:15:11.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:15:11.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:11.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:11.220 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:11.220 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:11.220 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:11.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:11.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:15:11.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:15:11.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:15:11.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:15:11.342 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:15:11.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:15:12.354 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:15:12.355 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:12.355 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:12.356 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:12.356 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.356 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:12.356 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:15:12.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:15:12.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:15:12.410 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:15:12.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:15:12.419 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:15:10.975+0000 7f853ae75dc0 0 load: jerasure load: lrc 2026-02-19T13:15:12.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects td/test-erasure-eio 2026-02-19T13:15:12.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:471: TEST_ec_recovery_multiple_objects: local dir=td/test-erasure-eio 2026-02-19T13:15:12.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:472: TEST_ec_recovery_multiple_objects: local objname=myobject 2026-02-19T13:15:12.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:474: TEST_ec_recovery_multiple_objects: ORIG_ARGS='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:15:12.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:475: TEST_ec_recovery_multiple_objects: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:12.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:476: TEST_ec_recovery_multiple_objects: setup_osds 7 2026-02-19T13:15:12.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-19T13:15:12.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:15:12.421 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-19T13:15:12.424 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:12.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:12.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:12.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7859283f-b10b-42cf-8778-914d76688f54 2026-02-19T13:15:12.428 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 7859283f-b10b-42cf-8778-914d76688f54 2026-02-19T13:15:12.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 7859283f-b10b-42cf-8778-914d76688f54' 2026-02-19T13:15:12.428 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:12.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDgDJdpXiYMGhAASEdn3Zwbc1HSOrSlc6ZnVw== 2026-02-19T13:15:12.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDgDJdpXiYMGhAASEdn3Zwbc1HSOrSlc6ZnVw=="}' 2026-02-19T13:15:12.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7859283f-b10b-42cf-8778-914d76688f54 -i td/test-erasure-eio/0/new.json 2026-02-19T13:15:12.658 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:12.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:15:12.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDgDJdpXiYMGhAASEdn3Zwbc1HSOrSlc6ZnVw== --osd-uuid 7859283f-b10b-42cf-8778-914d76688f54 2026-02-19T13:15:12.686 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:12.687+0000 7fe0f83ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:12.688 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:12.688+0000 7fe0f83ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:12.689 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:12.689+0000 7fe0f83ad900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:12.689 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:12.689+0000 7fe0f83ad900 -1 bdev(0x562e4c572c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:12.689 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:12.689+0000 7fe0f83ad900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:15:13.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:15:13.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:13.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:15:13.131 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:15:13.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:13.432 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:15:13.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:15:13.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:13.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:13.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:13.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:13.449 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:13.450+0000 7efd539ce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:13.451 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:13.451+0000 7efd539ce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:13.452 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:13.452+0000 7efd539ce900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:13.645 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:13.645+0000 7efd539ce900 -1 Falling back to public interface 2026-02-19T13:15:13.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:15:13.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:13.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:15:13.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:13.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:13.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:13.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:13.655 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:13.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:13.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:13.975 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:13.975+0000 7efd539ce900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:15:14.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:14.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:14.875 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:14.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:14.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:14.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:15.096 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1627037243,v1:127.0.0.1:6803/1627037243] [v2:127.0.0.1:6804/1627037243,v1:127.0.0.1:6805/1627037243] exists,up 7859283f-b10b-42cf-8778-914d76688f54 2026-02-19T13:15:15.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:15.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:15.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:15.101 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:15.102 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:15.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:15.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:15:15.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:15.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2cdb3124-d886-4a33-bea7-1a5ed87e332e 2026-02-19T13:15:15.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 2cdb3124-d886-4a33-bea7-1a5ed87e332e' 2026-02-19T13:15:15.104 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 2cdb3124-d886-4a33-bea7-1a5ed87e332e 2026-02-19T13:15:15.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:15.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDjDJdp/ba6BhAAi6JNCRJghVFHumUMBBI4ZA== 2026-02-19T13:15:15.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDjDJdp/ba6BhAAi6JNCRJghVFHumUMBBI4ZA=="}' 2026-02-19T13:15:15.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2cdb3124-d886-4a33-bea7-1a5ed87e332e -i td/test-erasure-eio/1/new.json 2026-02-19T13:15:15.334 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:15.345 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:15:15.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDjDJdp/ba6BhAAi6JNCRJghVFHumUMBBI4ZA== --osd-uuid 2cdb3124-d886-4a33-bea7-1a5ed87e332e 2026-02-19T13:15:15.363 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:15.363+0000 7f851bf6a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:15.364 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:15.365+0000 7f851bf6a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:15.365 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:15.365+0000 7f851bf6a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:15.366 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:15.366+0000 7f851bf6a900 -1 bdev(0x55cee7e66c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:15.366 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:15.366+0000 7f851bf6a900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:15:15.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:15:15.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:15.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:15:15.822 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:15:15.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:16.122 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:15:16.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:15:16.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:16.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:16.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:16.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:16.139 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:16.139+0000 7ff3c2606900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:16.140 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:16.141+0000 7ff3c2606900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:16.141 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:16.141+0000 7ff3c2606900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:16.291 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:16.291+0000 7ff3c2606900 -1 Falling back to public interface 2026-02-19T13:15:16.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:15:16.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:16.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:15:16.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:16.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:16.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:16.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:16.348 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:16.349 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:16.349 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:15:16.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:16.688 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:16.689+0000 7ff3c2606900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:15:17.573 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:17.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:17.574 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:17.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:17.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:17.575 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:15:17.801 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3152654598,v1:127.0.0.1:6811/3152654598] [v2:127.0.0.1:6812/3152654598,v1:127.0.0.1:6813/3152654598] exists,up 2cdb3124-d886-4a33-bea7-1a5ed87e332e 2026-02-19T13:15:17.805 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:17.805 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:17.805 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:17.805 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:17.805 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:15:17.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:17.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:17.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a77bded8-d86a-43d0-8dfe-5f549cb01450 2026-02-19T13:15:17.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 a77bded8-d86a-43d0-8dfe-5f549cb01450' 2026-02-19T13:15:17.809 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 a77bded8-d86a-43d0-8dfe-5f549cb01450 2026-02-19T13:15:17.809 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:17.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDlDJdpNJnHMBAAPFngz1r99Nl2nCfXtpTIRg== 2026-02-19T13:15:17.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDlDJdpNJnHMBAAPFngz1r99Nl2nCfXtpTIRg=="}' 2026-02-19T13:15:17.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a77bded8-d86a-43d0-8dfe-5f549cb01450 -i td/test-erasure-eio/2/new.json 2026-02-19T13:15:18.041 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:15:18.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:15:18.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDlDJdpNJnHMBAAPFngz1r99Nl2nCfXtpTIRg== --osd-uuid a77bded8-d86a-43d0-8dfe-5f549cb01450 2026-02-19T13:15:18.069 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:18.070+0000 7f8593b12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:18.070 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:18.071+0000 7f8593b12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:18.071 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:18.072+0000 7f8593b12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:18.071 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:18.072+0000 7f8593b12900 -1 bdev(0x55a4feefcc00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:18.071 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:18.072+0000 7f8593b12900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:15:18.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:15:18.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:18.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:15:18.505 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:15:18.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:18.811 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:15:18.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:15:18.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:18.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:18.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:18.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:18.827 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:18.828+0000 7f78362ba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:18.828 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:18.829+0000 7f78362ba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:18.829 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:18.830+0000 7f78362ba900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:19.026 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:19.027+0000 7f78362ba900 -1 Falling back to public interface 2026-02-19T13:15:19.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:15:19.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:19.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:15:19.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:19.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:19.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:19.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:19.037 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:19.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:19.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:15:19.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:19.421 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:19.423+0000 7f78362ba900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:15:20.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:20.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:20.257 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:20.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:20.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:20.259 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:15:20.481 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2822551295,v1:127.0.0.1:6819/2822551295] [v2:127.0.0.1:6820/2822551295,v1:127.0.0.1:6821/2822551295] exists,up a77bded8-d86a-43d0-8dfe-5f549cb01450 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:15:20.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:20.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=2a8077fb-22d2-4974-bccc-8dbb0edd74ce 2026-02-19T13:15:20.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 2a8077fb-22d2-4974-bccc-8dbb0edd74ce' 2026-02-19T13:15:20.486 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 2a8077fb-22d2-4974-bccc-8dbb0edd74ce 2026-02-19T13:15:20.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:20.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDoDJdp3fGsHRAA5InGwEZ1d7aFHgFIk26dWg== 2026-02-19T13:15:20.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDoDJdp3fGsHRAA5InGwEZ1d7aFHgFIk26dWg=="}' 2026-02-19T13:15:20.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 2a8077fb-22d2-4974-bccc-8dbb0edd74ce -i td/test-erasure-eio/3/new.json 2026-02-19T13:15:20.720 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:15:20.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:15:20.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDoDJdp3fGsHRAA5InGwEZ1d7aFHgFIk26dWg== --osd-uuid 2a8077fb-22d2-4974-bccc-8dbb0edd74ce 2026-02-19T13:15:20.750 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:20.751+0000 7f34637dd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:20.751 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:20.753+0000 7f34637dd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:20.752 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:20.753+0000 7f34637dd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:20.752 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:20.754+0000 7f34637dd900 -1 bdev(0x555a34604c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:20.752 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:20.754+0000 7f34637dd900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:15:21.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:15:21.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:21.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:15:21.205 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:15:21.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:21.508 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:15:21.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:15:21.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:21.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:21.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:21.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:21.525 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:21.527+0000 7efdf1700900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:21.527 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:21.528+0000 7efdf1700900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:21.528 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:21.529+0000 7efdf1700900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:21.722 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:21.724+0000 7efdf1700900 -1 Falling back to public interface 2026-02-19T13:15:21.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:15:21.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:21.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:15:21.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:21.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:21.733 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:21.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:21.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:21.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:21.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:15:21.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:22.118 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:22.119+0000 7efdf1700900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:15:22.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:22.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:22.959 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:22.960 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:22.960 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:22.960 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:15:23.180 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1226725553,v1:127.0.0.1:6827/1226725553] [v2:127.0.0.1:6828/1226725553,v1:127.0.0.1:6829/1226725553] exists,up 2a8077fb-22d2-4974-bccc-8dbb0edd74ce 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T13:15:23.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:23.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:23.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=18075192-2e93-4d42-8e15-c188c5d30b49 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 18075192-2e93-4d42-8e15-c188c5d30b49' 2026-02-19T13:15:23.188 INFO:tasks.workunit.client.0.trial111.stdout:add osd4 18075192-2e93-4d42-8e15-c188c5d30b49 2026-02-19T13:15:23.189 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:23.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDrDJdpmXXJCxAAHYMQf/ph6n1Xk3Ne8wV3Cw== 2026-02-19T13:15:23.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDrDJdpmXXJCxAAHYMQf/ph6n1Xk3Ne8wV3Cw=="}' 2026-02-19T13:15:23.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 18075192-2e93-4d42-8e15-c188c5d30b49 -i td/test-erasure-eio/4/new.json 2026-02-19T13:15:23.418 INFO:tasks.workunit.client.0.trial111.stdout:4 2026-02-19T13:15:23.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T13:15:23.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDrDJdpmXXJCxAAHYMQf/ph6n1Xk3Ne8wV3Cw== --osd-uuid 18075192-2e93-4d42-8e15-c188c5d30b49 2026-02-19T13:15:23.449 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:23.450+0000 7fc73c042900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:23.450 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:23.452+0000 7fc73c042900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:23.451 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:23.453+0000 7fc73c042900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:23.452 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:23.453+0000 7fc73c042900 -1 bdev(0x55eadf49ac00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:23.452 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:23.453+0000 7fc73c042900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T13:15:23.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T13:15:23.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:23.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T13:15:23.882 INFO:tasks.workunit.client.0.trial111.stdout:adding osd4 key to auth repository 2026-02-19T13:15:23.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:24.183 INFO:tasks.workunit.client.0.trial111.stdout:start osd.4 2026-02-19T13:15:24.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T13:15:24.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:24.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:24.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:24.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:24.199 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:24.199+0000 7f38b5430900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:24.200 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:24.201+0000 7f38b5430900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:24.201 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:24.202+0000 7f38b5430900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:24.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T13:15:24.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:24.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T13:15:24.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:24.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:24.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:24.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:24.407 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:24.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:24.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:15:24.436 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:24.437+0000 7f38b5430900 -1 Falling back to public interface 2026-02-19T13:15:24.630 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:24.766 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:24.766+0000 7f38b5430900 -1 osd.4 0 log_to_monitors true 2026-02-19T13:15:25.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:25.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:25.632 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:25.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:25.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:25.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:15:25.856 INFO:tasks.workunit.client.0.trial111.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/1483403142,v1:127.0.0.1:6835/1483403142] [v2:127.0.0.1:6836/1483403142,v1:127.0.0.1:6837/1483403142] exists,up 18075192-2e93-4d42-8e15-c188c5d30b49 2026-02-19T13:15:25.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:25.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:25.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:25.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:25.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T13:15:25.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:25.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d6de1dd1-fac3-42d9-b09b-5eda0503ea45 2026-02-19T13:15:25.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 d6de1dd1-fac3-42d9-b09b-5eda0503ea45' 2026-02-19T13:15:25.864 INFO:tasks.workunit.client.0.trial111.stdout:add osd5 d6de1dd1-fac3-42d9-b09b-5eda0503ea45 2026-02-19T13:15:25.864 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:25.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDtDJdpksQONBAAyhgpCt5sA/UhRV81mK7r8g== 2026-02-19T13:15:25.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDtDJdpksQONBAAyhgpCt5sA/UhRV81mK7r8g=="}' 2026-02-19T13:15:25.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d6de1dd1-fac3-42d9-b09b-5eda0503ea45 -i td/test-erasure-eio/5/new.json 2026-02-19T13:15:26.092 INFO:tasks.workunit.client.0.trial111.stdout:5 2026-02-19T13:15:26.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T13:15:26.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDtDJdpksQONBAAyhgpCt5sA/UhRV81mK7r8g== --osd-uuid d6de1dd1-fac3-42d9-b09b-5eda0503ea45 2026-02-19T13:15:26.121 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:26.121+0000 7f0b7a9c5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:26.123 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:26.123+0000 7f0b7a9c5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:26.124 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:26.124+0000 7f0b7a9c5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:26.124 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:26.124+0000 7f0b7a9c5900 -1 bdev(0x55f20a538c00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:26.124 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:26.124+0000 7f0b7a9c5900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T13:15:26.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T13:15:26.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:26.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T13:15:26.554 INFO:tasks.workunit.client.0.trial111.stdout:adding osd5 key to auth repository 2026-02-19T13:15:26.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:26.856 INFO:tasks.workunit.client.0.trial111.stdout:start osd.5 2026-02-19T13:15:26.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T13:15:26.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:26.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:26.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:26.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:26.874 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:26.874+0000 7f655a8b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:26.875 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:26.875+0000 7f655a8b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:26.876 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:26.876+0000 7f655a8b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:27.073 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:27.073+0000 7f655a8b6900 -1 Falling back to public interface 2026-02-19T13:15:27.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T13:15:27.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:27.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T13:15:27.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:27.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:27.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:27.084 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:27.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:27.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:27.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:15:27.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:27.460 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:27.460+0000 7f655a8b6900 -1 osd.5 0 log_to_monitors true 2026-02-19T13:15:28.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:28.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:28.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:28.305 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:28.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:28.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:15:28.530 INFO:tasks.workunit.client.0.trial111.stdout:osd.5 up in weight 1 up_from 44 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/973955724,v1:127.0.0.1:6843/973955724] [v2:127.0.0.1:6844/973955724,v1:127.0.0.1:6845/973955724] exists,up d6de1dd1-fac3-42d9-b09b-5eda0503ea45 2026-02-19T13:15:28.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:28.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T13:15:28.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:28.536 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:28.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=68361fe2-b185-4250-833a-3807e06b8c95 2026-02-19T13:15:28.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 68361fe2-b185-4250-833a-3807e06b8c95' 2026-02-19T13:15:28.539 INFO:tasks.workunit.client.0.trial111.stdout:add osd6 68361fe2-b185-4250-833a-3807e06b8c95 2026-02-19T13:15:28.539 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:28.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDwDJdpsXCWIBAA6CzdVBmnpgvGc6l88C+FHQ== 2026-02-19T13:15:28.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDwDJdpsXCWIBAA6CzdVBmnpgvGc6l88C+FHQ=="}' 2026-02-19T13:15:28.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 68361fe2-b185-4250-833a-3807e06b8c95 -i td/test-erasure-eio/6/new.json 2026-02-19T13:15:28.775 INFO:tasks.workunit.client.0.trial111.stdout:6 2026-02-19T13:15:28.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T13:15:28.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDwDJdpsXCWIBAA6CzdVBmnpgvGc6l88C+FHQ== --osd-uuid 68361fe2-b185-4250-833a-3807e06b8c95 2026-02-19T13:15:28.803 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:28.803+0000 7fb039f17900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:28.804 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:28.805+0000 7fb039f17900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:28.805 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:28.805+0000 7fb039f17900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:28.806 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:28.806+0000 7fb039f17900 -1 bdev(0x55639482ac00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:28.806 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:28.806+0000 7fb039f17900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T13:15:29.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T13:15:29.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:29.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T13:15:29.236 INFO:tasks.workunit.client.0.trial111.stdout:adding osd6 key to auth repository 2026-02-19T13:15:29.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:29.535 INFO:tasks.workunit.client.0.trial111.stdout:start osd.6 2026-02-19T13:15:29.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T13:15:29.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:29.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:29.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:29.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:29.552 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:29.552+0000 7f735b126900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:29.553 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:29.553+0000 7f735b126900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:29.554 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:29.554+0000 7f735b126900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:29.735 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:29.735+0000 7f735b126900 -1 Falling back to public interface 2026-02-19T13:15:29.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T13:15:29.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:29.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T13:15:29.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:29.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:29.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:29.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:29.765 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:29.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:29.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:15:29.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:30.116 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:30.116+0000 7f735b126900 -1 osd.6 0 log_to_monitors true 2026-02-19T13:15:30.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:30.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:30.987 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:30.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:30.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:30.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:15:31.214 INFO:tasks.workunit.client.0.trial111.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1094607107,v1:127.0.0.1:6851/1094607107] [v2:127.0.0.1:6852/1094607107,v1:127.0.0.1:6853/1094607107] exists,up 68361fe2-b185-4250-833a-3807e06b8c95 2026-02-19T13:15:31.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:31.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:31.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:31.216 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:15:31.216 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:15:31.216 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:15:31.216 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:31.216 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:31.217 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:31.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:15:31.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:15:31.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:15:31.274 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:15:31.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:15:31.284 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:15:13.648+0000 7efd539ce900 0 load: jerasure load: lrc 2026-02-19T13:15:31.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:477: TEST_ec_recovery_multiple_objects: CEPH_ARGS='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:15:31.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:479: TEST_ec_recovery_multiple_objects: local poolname=pool-jerasure 2026-02-19T13:15:31.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:480: TEST_ec_recovery_multiple_objects: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T13:15:31.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:15:31.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:15:31.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T13:15:31.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:15:31.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T13:15:31.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:15:31.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T13:15:31.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:15:31.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:15:31.957 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:15:31.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:15:32.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:15:32.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:15:32.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:15:32.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:15:32.970 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:15:32.971 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:32.971 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:15:32.971 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:32.971 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:32.971 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:33.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:15:33.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:15:33.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:15:33.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:15:33.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:33.051 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:33.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:15:33.270 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:15:33.270 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:15:33.270 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:15:33.270 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:15:33.271 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:15:33.271 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:15:33.271 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:33.271 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:33.271 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:33.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803783 2026-02-19T13:15:33.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803783 2026-02-19T13:15:33.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783' 2026-02-19T13:15:33.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:33.348 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:15:33.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-19T13:15:33.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-19T13:15:33.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607558' 2026-02-19T13:15:33.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:33.429 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:15:33.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T13:15:33.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T13:15:33.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-77309411333' 2026-02-19T13:15:33.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:33.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:15:33.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-19T13:15:33.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-19T13:15:33.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-77309411333 3-111669149701' 2026-02-19T13:15:33.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:33.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:15:33.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855365 2026-02-19T13:15:33.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855365 2026-02-19T13:15:33.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-77309411333 3-111669149701 4-150323855365' 2026-02-19T13:15:33.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:33.667 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:15:33.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561028 2026-02-19T13:15:33.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561028 2026-02-19T13:15:33.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-77309411333 3-111669149701 4-150323855365 5-188978561028' 2026-02-19T13:15:33.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:33.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:15:33.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299395 2026-02-19T13:15:33.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299395 2026-02-19T13:15:33.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803783 1-51539607558 2-77309411333 3-111669149701 4-150323855365 5-188978561028 6-223338299395' 2026-02-19T13:15:33.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:33.829 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803783 2026-02-19T13:15:33.829 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:33.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:33.830 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803783 2026-02-19T13:15:33.831 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:33.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803783 2026-02-19T13:15:33.832 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803783 2026-02-19T13:15:33.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803783' 2026-02-19T13:15:33.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:34.056 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803783 2026-02-19T13:15:34.057 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:34.057 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-19T13:15:34.057 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:34.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:15:34.058 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607558 2026-02-19T13:15:34.059 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:34.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607558 2026-02-19T13:15:34.060 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 51539607558 2026-02-19T13:15:34.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-19T13:15:34.060 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:15:34.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-19T13:15:34.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:34.283 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T13:15:34.283 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:34.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:15:34.284 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411333 2026-02-19T13:15:34.285 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:34.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411333 2026-02-19T13:15:34.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T13:15:34.286 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411333 2026-02-19T13:15:34.286 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:15:34.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T13:15:34.508 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:34.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-19T13:15:34.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:34.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:15:34.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149701 2026-02-19T13:15:34.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:34.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149701 2026-02-19T13:15:34.511 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149701 2026-02-19T13:15:34.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-19T13:15:34.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:15:34.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-19T13:15:34.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:34.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855365 2026-02-19T13:15:34.732 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:34.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:15:34.733 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855365 2026-02-19T13:15:34.733 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:34.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855365 2026-02-19T13:15:34.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855365' 2026-02-19T13:15:34.735 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 150323855365 2026-02-19T13:15:34.735 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:15:34.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855365 -lt 150323855365 2026-02-19T13:15:34.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:34.958 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561028 2026-02-19T13:15:34.958 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:34.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:15:34.959 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561028 2026-02-19T13:15:34.960 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:34.961 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561028 2026-02-19T13:15:34.961 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561028' 2026-02-19T13:15:34.961 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 188978561028 2026-02-19T13:15:34.961 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:15:35.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561026 -lt 188978561028 2026-02-19T13:15:35.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:36.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:36.182 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:15:36.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-19T13:15:36.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:36.403 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299395 2026-02-19T13:15:36.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:36.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:15:36.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299395 2026-02-19T13:15:36.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:36.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299395 2026-02-19T13:15:36.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299395' 2026-02-19T13:15:36.407 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 223338299395 2026-02-19T13:15:36.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:15:36.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299395 -lt 223338299395 2026-02-19T13:15:36.635 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:36.636 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:36.636 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:36.932 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:15:36.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:36.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:36.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:36.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:36.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:36.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:36.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:37.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:15:37.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:37.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:37.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:37.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:15:37.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:37.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:37.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:482: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-19T13:15:37.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:15:37.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:15:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-19T13:15:37.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:483: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-19T13:15:37.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:15:37.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:15:37.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-19T13:15:37.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:15:37.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:15:37.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:15:37.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:15:37.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:484: TEST_ec_recovery_multiple_objects: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:15:37.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:15:37.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:15:37.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-19T13:15:37.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:486: TEST_ec_recovery_multiple_objects: ceph osd out 0 2026-02-19T13:15:37.772 INFO:tasks.workunit.client.0.trial111.stderr:osd.0 is already out. 2026-02-19T13:15:37.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:489: TEST_ec_recovery_multiple_objects: wait_for_clean 2026-02-19T13:15:37.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:15:37.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:15:37.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:15:37.785 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:15:37.785 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:15:37.785 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:15:37.785 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:15:37.786 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:15:37.786 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:15:37.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:15:37.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:15:37.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:15:37.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:15:37.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:15:37.865 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:15:38.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:15:38.086 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:15:38.086 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:15:38.086 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:15:38.086 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:15:38.086 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:15:38.086 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:15:38.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:15:38.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:38.087 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:15:38.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803786 2026-02-19T13:15:38.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803786 2026-02-19T13:15:38.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786' 2026-02-19T13:15:38.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:38.168 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:15:38.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-19T13:15:38.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-19T13:15:38.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561' 2026-02-19T13:15:38.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:38.245 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:15:38.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-19T13:15:38.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-19T13:15:38.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411336' 2026-02-19T13:15:38.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:38.325 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:15:38.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149704 2026-02-19T13:15:38.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149704 2026-02-19T13:15:38.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411336 3-111669149704' 2026-02-19T13:15:38.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:38.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:15:38.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855368 2026-02-19T13:15:38.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855368 2026-02-19T13:15:38.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411336 3-111669149704 4-150323855368' 2026-02-19T13:15:38.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:38.485 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:15:38.562 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561030 2026-02-19T13:15:38.562 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561030 2026-02-19T13:15:38.562 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411336 3-111669149704 4-150323855368 5-188978561030' 2026-02-19T13:15:38.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:15:38.563 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:15:38.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299398 2026-02-19T13:15:38.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299398 2026-02-19T13:15:38.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803786 1-51539607561 2-77309411336 3-111669149704 4-150323855368 5-188978561030 6-223338299398' 2026-02-19T13:15:38.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:38.641 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803786 2026-02-19T13:15:38.641 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:38.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:15:38.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803786 2026-02-19T13:15:38.643 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:38.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803786 2026-02-19T13:15:38.643 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803786 2026-02-19T13:15:38.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803786' 2026-02-19T13:15:38.644 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:38.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803786 2026-02-19T13:15:38.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:15:39.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:15:39.865 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:15:40.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803786 -lt 25769803786 2026-02-19T13:15:40.085 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:40.086 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-19T13:15:40.086 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:40.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:15:40.087 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607561 2026-02-19T13:15:40.087 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:40.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607561 2026-02-19T13:15:40.089 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 51539607561 2026-02-19T13:15:40.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-19T13:15:40.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:15:40.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-19T13:15:40.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:40.315 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-19T13:15:40.315 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:40.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:15:40.316 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411336 2026-02-19T13:15:40.316 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:40.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411336 2026-02-19T13:15:40.318 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411336 2026-02-19T13:15:40.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-19T13:15:40.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:15:40.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-19T13:15:40.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:40.543 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149704 2026-02-19T13:15:40.543 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:40.543 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:15:40.544 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149704 2026-02-19T13:15:40.544 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:40.545 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149704 2026-02-19T13:15:40.545 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149704 2026-02-19T13:15:40.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149704' 2026-02-19T13:15:40.546 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:15:40.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149704 -lt 111669149704 2026-02-19T13:15:40.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:40.767 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855368 2026-02-19T13:15:40.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:40.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:15:40.769 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855368 2026-02-19T13:15:40.769 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:40.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855368 2026-02-19T13:15:40.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855368' 2026-02-19T13:15:40.770 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 150323855368 2026-02-19T13:15:40.771 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:15:40.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855368 -lt 150323855368 2026-02-19T13:15:40.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:40.995 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561030 2026-02-19T13:15:40.995 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:40.995 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:15:40.996 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561030 2026-02-19T13:15:40.996 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:40.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561030 2026-02-19T13:15:40.997 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 188978561030 2026-02-19T13:15:40.998 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561030' 2026-02-19T13:15:40.998 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:15:41.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561031 -lt 188978561030 2026-02-19T13:15:41.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:15:41.216 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299398 2026-02-19T13:15:41.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:15:41.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:15:41.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299398 2026-02-19T13:15:41.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:15:41.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299398 2026-02-19T13:15:41.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299398' 2026-02-19T13:15:41.220 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 223338299398 2026-02-19T13:15:41.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:15:41.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299398 -lt 223338299398 2026-02-19T13:15:41.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:15:41.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:41.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:41.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:15:41.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:15:41.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:15:41.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:15:41.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:15:41.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:15:41.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:15:41.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:15:41.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:15:41.969 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:15:41.969 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:15:41.970 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:491: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:15:42.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-19T13:15:42.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:15:42.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:15:42.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:492: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-19T13:15:42.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:15:42.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:15:42.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-19T13:15:42.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:15:42.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:15:42.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-19T13:15:42.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:15:42.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:15:42.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:493: TEST_ec_recovery_multiple_objects: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-19T13:15:42.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:15:42.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:15:42.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-19T13:15:42.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:15:42.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:15:42.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-19T13:15:42.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:15:42.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:15:42.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:495: TEST_ec_recovery_multiple_objects: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:15:42.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:15:42.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:15:42.582 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:15:42.594 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:15:42.861 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:15:42.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:15:42.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:15:42.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:15:42.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:15:42.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:42.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:42.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:42.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:42.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:42.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:42.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:42.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:42.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:42.984 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:42.984 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:42.985 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:42.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:42.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:42.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:42.986 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:42.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:42.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:15:42.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:15:43.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:43.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:43.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:43.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:15:43.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:43.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:43.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:15:43.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:15:43.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:15:43.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:15:43.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:15:43.007 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:15:43.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:15:43.008 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:15:43.009 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:15:43.009 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:15:43.009 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:15:43.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:43.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:15:43.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:15:43.011 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:15:43.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:15:43.012 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:15:43.012 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:15:43.013 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:15:43.013 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:43.013 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:43.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:15:43.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:15:43.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:15:43.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:15:43.016 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:15:43.016 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:43.016 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:43.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:15:43.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:15:43.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:15:43.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:15:43.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:15:43.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:15:43.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:15:43.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:15:43.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:15:43.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:15:43.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:15:43.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:15:43.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:15:43.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:43.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:43.038 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:43.038 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:43.038 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:43.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:43.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:15:43.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:15:43.064 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:15:43.064 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:15:43.064 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:15:43.064 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:15:43.064 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:15:43.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:15:43.124 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:15:43.124 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:15:43.125 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:15:43.125 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:15:43.125 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:15:43.125 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:15:43.125 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:43.125 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:43.126 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:43.126 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:43.126 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:43.126 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:15:43.126 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:15:43.126 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:15:43.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:15:43.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:15:43.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:15:43.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:15:43.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:15:43.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:15:43.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:15:43.300 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:15:43.300 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:43.300 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:43.301 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:43.301 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:43.301 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:43.301 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:43.301 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:15:43.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:15:43.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:15:43.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:15:43.421 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:15:43.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:15:44.434 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:15:44.434 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:15:44.434 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:15:44.434 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:15:44.434 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:44.435 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:44.435 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:15:44.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:15:44.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:15:44.488 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:15:44.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:15:44.496 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:15:43.055+0000 7f38e1c9ddc0 0 load: jerasure load: lrc 2026-02-19T13:15:44.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_multiple_objects_eio td/test-erasure-eio 2026-02-19T13:15:44.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:500: TEST_ec_recovery_multiple_objects_eio: local dir=td/test-erasure-eio 2026-02-19T13:15:44.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:501: TEST_ec_recovery_multiple_objects_eio: local objname=myobject 2026-02-19T13:15:44.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:503: TEST_ec_recovery_multiple_objects_eio: ORIG_ARGS='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:15:44.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:504: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:44.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:505: TEST_ec_recovery_multiple_objects_eio: setup_osds 7 2026-02-19T13:15:44.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-19T13:15:44.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:15:44.498 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-19T13:15:44.502 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-19T13:15:44.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:44.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:15:44.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:44.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:44.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:44.503 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:44.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ac9dd9a8-45ad-4e3a-89dd-a858f93d58f0 2026-02-19T13:15:44.505 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 ac9dd9a8-45ad-4e3a-89dd-a858f93d58f0 2026-02-19T13:15:44.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 ac9dd9a8-45ad-4e3a-89dd-a858f93d58f0' 2026-02-19T13:15:44.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:44.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAADZdpThCrHhAAF9DlUUtw1HzOZS1hy4UjQQ== 2026-02-19T13:15:44.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAADZdpThCrHhAAF9DlUUtw1HzOZS1hy4UjQQ=="}' 2026-02-19T13:15:44.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ac9dd9a8-45ad-4e3a-89dd-a858f93d58f0 -i td/test-erasure-eio/0/new.json 2026-02-19T13:15:44.734 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:44.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:15:44.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAADZdpThCrHhAAF9DlUUtw1HzOZS1hy4UjQQ== --osd-uuid ac9dd9a8-45ad-4e3a-89dd-a858f93d58f0 2026-02-19T13:15:44.764 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:44.764+0000 7f4f8064e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:44.765 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:44.765+0000 7f4f8064e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:44.766 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:44.766+0000 7f4f8064e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:44.766 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:44.766+0000 7f4f8064e900 -1 bdev(0x56528a578c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:44.766 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:44.766+0000 7f4f8064e900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:15:45.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:15:45.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:45.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:15:45.189 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:15:45.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:45.489 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:15:45.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:15:45.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:45.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:45.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:45.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:45.504 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:45.505+0000 7f0ea186e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:45.506 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:45.506+0000 7f0ea186e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:45.507 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:45.507+0000 7f0ea186e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:45.644 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:45.644+0000 7f0ea186e900 -1 Falling back to public interface 2026-02-19T13:15:45.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:15:45.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:45.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:15:45.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:45.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:45.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:45.713 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:45.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:45.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:45.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:45.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:45.993 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:45.993+0000 7f0ea186e900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:15:46.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:46.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:46.940 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:46.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:46.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:46.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:15:47.168 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3316012453,v1:127.0.0.1:6803/3316012453] [v2:127.0.0.1:6804/3316012453,v1:127.0.0.1:6805/3316012453] exists,up ac9dd9a8-45ad-4e3a-89dd-a858f93d58f0 2026-02-19T13:15:47.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:47.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:47.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:47.174 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:47.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:47.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:15:47.176 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:47.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eca41f23-6f18-46ab-af8d-2499bd24e899 2026-02-19T13:15:47.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 eca41f23-6f18-46ab-af8d-2499bd24e899' 2026-02-19T13:15:47.176 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 eca41f23-6f18-46ab-af8d-2499bd24e899 2026-02-19T13:15:47.176 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:47.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQADDZdpZEkDCxAAx+jFype0JYEMiJmeqgsr4Q== 2026-02-19T13:15:47.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQADDZdpZEkDCxAAx+jFype0JYEMiJmeqgsr4Q=="}' 2026-02-19T13:15:47.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eca41f23-6f18-46ab-af8d-2499bd24e899 -i td/test-erasure-eio/1/new.json 2026-02-19T13:15:47.405 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:47.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:15:47.418 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQADDZdpZEkDCxAAx+jFype0JYEMiJmeqgsr4Q== --osd-uuid eca41f23-6f18-46ab-af8d-2499bd24e899 2026-02-19T13:15:47.435 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:47.436+0000 7fad171dd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:47.437 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:47.437+0000 7fad171dd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:47.438 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:47.438+0000 7fad171dd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:47.438 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:47.438+0000 7fad171dd900 -1 bdev(0x5567de91cc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:47.438 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:47.438+0000 7fad171dd900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:15:47.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:15:47.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:47.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:15:47.841 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:15:47.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:48.140 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:15:48.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:15:48.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:48.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:48.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:48.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:48.155 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:48.156+0000 7fa6ddb24900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:48.157 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:48.157+0000 7fa6ddb24900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:48.158 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:48.158+0000 7fa6ddb24900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:48.333 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:48.333+0000 7fa6ddb24900 -1 Falling back to public interface 2026-02-19T13:15:48.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:15:48.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:48.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:15:48.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:48.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:48.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:48.363 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:48.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:48.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:48.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:15:48.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:48.718 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:48.718+0000 7fa6ddb24900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:15:49.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:49.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:49.587 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:49.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:49.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:49.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:15:49.812 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/135229299,v1:127.0.0.1:6811/135229299] [v2:127.0.0.1:6812/135229299,v1:127.0.0.1:6813/135229299] exists,up eca41f23-6f18-46ab-af8d-2499bd24e899 2026-02-19T13:15:49.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:49.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:49.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:49.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:49.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:15:49.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:49.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5fefd613-4774-465c-a553-8fa0c1818d0a 2026-02-19T13:15:49.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5fefd613-4774-465c-a553-8fa0c1818d0a' 2026-02-19T13:15:49.820 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 5fefd613-4774-465c-a553-8fa0c1818d0a 2026-02-19T13:15:49.820 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:49.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAFDZdp6QdjMRAAof0qQeyHztJ88t812dn2tg== 2026-02-19T13:15:49.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAFDZdp6QdjMRAAof0qQeyHztJ88t812dn2tg=="}' 2026-02-19T13:15:49.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5fefd613-4774-465c-a553-8fa0c1818d0a -i td/test-erasure-eio/2/new.json 2026-02-19T13:15:50.048 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:15:50.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:15:50.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAFDZdp6QdjMRAAof0qQeyHztJ88t812dn2tg== --osd-uuid 5fefd613-4774-465c-a553-8fa0c1818d0a 2026-02-19T13:15:50.077 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:50.077+0000 7f931b081900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:50.078 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:50.079+0000 7f931b081900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:50.079 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:50.079+0000 7f931b081900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:50.080 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:50.080+0000 7f931b081900 -1 bdev(0x56236821ac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:50.080 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:50.080+0000 7f931b081900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:15:50.545 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:15:50.545 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:50.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:15:50.547 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:15:50.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:50.848 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:15:50.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:15:50.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:50.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:50.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:50.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:50.864 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:50.864+0000 7f6101627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:50.865 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:50.865+0000 7f6101627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:50.866 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:50.866+0000 7f6101627900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:51.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:15:51.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:51.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:15:51.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:51.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:51.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:51.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:51.070 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:51.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:51.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:15:51.100 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:51.101+0000 7f6101627900 -1 Falling back to public interface 2026-02-19T13:15:51.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:51.532 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:51.532+0000 7f6101627900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:15:52.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:52.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:52.298 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:52.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:52.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:52.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:15:52.520 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2557327209,v1:127.0.0.1:6819/2557327209] [v2:127.0.0.1:6820/2557327209,v1:127.0.0.1:6821/2557327209] exists,up 5fefd613-4774-465c-a553-8fa0c1818d0a 2026-02-19T13:15:52.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:52.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:52.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:52.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:52.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:15:52.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:52.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:52.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d887151d-e600-4b5c-b099-0219321119bf 2026-02-19T13:15:52.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 d887151d-e600-4b5c-b099-0219321119bf' 2026-02-19T13:15:52.528 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 d887151d-e600-4b5c-b099-0219321119bf 2026-02-19T13:15:52.528 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:52.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAIDZdpiqP+HxAA+ws/4uWHEqH5HIN9dTl/JQ== 2026-02-19T13:15:52.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAIDZdpiqP+HxAA+ws/4uWHEqH5HIN9dTl/JQ=="}' 2026-02-19T13:15:52.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d887151d-e600-4b5c-b099-0219321119bf -i td/test-erasure-eio/3/new.json 2026-02-19T13:15:52.758 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:15:52.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:15:52.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAIDZdpiqP+HxAA+ws/4uWHEqH5HIN9dTl/JQ== --osd-uuid d887151d-e600-4b5c-b099-0219321119bf 2026-02-19T13:15:52.787 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:52.788+0000 7fca1932a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:52.789 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:52.789+0000 7fca1932a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:52.790 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:52.790+0000 7fca1932a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:52.791 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:52.791+0000 7fca1932a900 -1 bdev(0x55b33ee2ec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:52.791 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:52.791+0000 7fca1932a900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:15:53.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:15:53.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:53.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:15:53.207 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:15:53.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:53.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:15:53.513 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:15:53.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:53.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:53.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:53.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:53.529 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:53.531+0000 7f03c5678900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:53.531 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:53.532+0000 7f03c5678900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:53.532 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:53.533+0000 7f03c5678900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:53.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:15:53.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:53.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:15:53.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:53.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:53.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:53.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:53.735 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:53.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:53.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:15:53.748 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:53.749+0000 7f03c5678900 -1 Falling back to public interface 2026-02-19T13:15:53.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:54.102 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:54.103+0000 7f03c5678900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:15:54.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:54.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:54.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:54.959 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:54.960 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:54.960 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:15:55.184 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3370881014,v1:127.0.0.1:6827/3370881014] [v2:127.0.0.1:6828/3370881014,v1:127.0.0.1:6829/3370881014] exists,up d887151d-e600-4b5c-b099-0219321119bf 2026-02-19T13:15:55.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:55.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T13:15:55.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:55.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T13:15:55.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:55.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:55.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:55.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:55.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T13:15:55.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:55.192 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:55.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:55.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:55.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:55.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:55.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:55.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T13:15:55.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:55.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=61e62958-f5cf-4068-a7a6-9c5f2e0fd69e 2026-02-19T13:15:55.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 61e62958-f5cf-4068-a7a6-9c5f2e0fd69e' 2026-02-19T13:15:55.195 INFO:tasks.workunit.client.0.trial111.stdout:add osd4 61e62958-f5cf-4068-a7a6-9c5f2e0fd69e 2026-02-19T13:15:55.195 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:55.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQALDZdpSkP+CxAAGweSFG3M7rld2QPYxDO7eQ== 2026-02-19T13:15:55.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQALDZdpSkP+CxAAGweSFG3M7rld2QPYxDO7eQ=="}' 2026-02-19T13:15:55.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 61e62958-f5cf-4068-a7a6-9c5f2e0fd69e -i td/test-erasure-eio/4/new.json 2026-02-19T13:15:55.422 INFO:tasks.workunit.client.0.trial111.stdout:4 2026-02-19T13:15:55.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T13:15:55.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQALDZdpSkP+CxAAGweSFG3M7rld2QPYxDO7eQ== --osd-uuid 61e62958-f5cf-4068-a7a6-9c5f2e0fd69e 2026-02-19T13:15:55.452 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:55.453+0000 7f8202740900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:55.453 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:55.454+0000 7f8202740900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:55.454 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:55.455+0000 7f8202740900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:55.454 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:55.456+0000 7f8202740900 -1 bdev(0x5627b5a7ac00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:55.455 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:55.456+0000 7f8202740900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T13:15:55.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T13:15:55.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:55.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T13:15:55.893 INFO:tasks.workunit.client.0.trial111.stdout:adding osd4 key to auth repository 2026-02-19T13:15:55.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:56.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T13:15:56.194 INFO:tasks.workunit.client.0.trial111.stdout:start osd.4 2026-02-19T13:15:56.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:56.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:56.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:56.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:56.209 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:56.210+0000 7f5d38f87900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:56.210 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:56.212+0000 7f5d38f87900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:56.211 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:56.213+0000 7f5d38f87900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:56.360 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:56.361+0000 7f5d38f87900 -1 Falling back to public interface 2026-02-19T13:15:56.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T13:15:56.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:56.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T13:15:56.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:56.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:56.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:56.417 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:56.417 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:56.418 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:56.418 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:15:56.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:56.698 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:56.699+0000 7f5d38f87900 -1 osd.4 0 log_to_monitors true 2026-02-19T13:15:57.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:15:57.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:57.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:15:57.640 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:15:57.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:57.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:15:57.861 INFO:tasks.workunit.client.0.trial111.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/2955191151,v1:127.0.0.1:6835/2955191151] [v2:127.0.0.1:6836/2955191151,v1:127.0.0.1:6837/2955191151] exists,up 61e62958-f5cf-4068-a7a6-9c5f2e0fd69e 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:15:57.865 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:15:57.866 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:15:57.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:15:57.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:15:57.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:15:57.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:15:57.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T13:15:57.868 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:15:57.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f0d17a3f-6114-4920-8826-310a8d56cf85 2026-02-19T13:15:57.869 INFO:tasks.workunit.client.0.trial111.stdout:add osd5 f0d17a3f-6114-4920-8826-310a8d56cf85 2026-02-19T13:15:57.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 f0d17a3f-6114-4920-8826-310a8d56cf85' 2026-02-19T13:15:57.869 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:15:57.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQANDZdpVEtXNBAAjs+xjT3b4anUGSasXOv3Pw== 2026-02-19T13:15:57.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQANDZdpVEtXNBAAjs+xjT3b4anUGSasXOv3Pw=="}' 2026-02-19T13:15:57.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f0d17a3f-6114-4920-8826-310a8d56cf85 -i td/test-erasure-eio/5/new.json 2026-02-19T13:15:58.098 INFO:tasks.workunit.client.0.trial111.stdout:5 2026-02-19T13:15:58.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T13:15:58.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQANDZdpVEtXNBAAjs+xjT3b4anUGSasXOv3Pw== --osd-uuid f0d17a3f-6114-4920-8826-310a8d56cf85 2026-02-19T13:15:58.126 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:58.128+0000 7fd7cf27a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:58.128 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:58.129+0000 7fd7cf27a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:58.128 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:58.130+0000 7fd7cf27a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:58.129 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:58.130+0000 7fd7cf27a900 -1 bdev(0x563f7f3eec00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T13:15:58.129 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:58.130+0000 7fd7cf27a900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T13:15:58.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T13:15:58.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:15:58.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T13:15:58.541 INFO:tasks.workunit.client.0.trial111.stdout:adding osd5 key to auth repository 2026-02-19T13:15:58.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:15:58.841 INFO:tasks.workunit.client.0.trial111.stdout:start osd.5 2026-02-19T13:15:58.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T13:15:58.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:15:58.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:15:58.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:15:58.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:15:58.858 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:58.859+0000 7f1a17140900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:58.860 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:58.861+0000 7f1a17140900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:58.861 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:58.862+0000 7f1a17140900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:15:59.043 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:59.044+0000 7f1a17140900 -1 Falling back to public interface 2026-02-19T13:15:59.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T13:15:59.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:15:59.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T13:15:59.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:15:59.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:15:59.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:15:59.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:15:59.066 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:15:59.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:15:59.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:15:59.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:15:59.466 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:15:59.467+0000 7f1a17140900 -1 osd.5 0 log_to_monitors true 2026-02-19T13:16:00.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:00.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:00.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:00.287 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:00.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:00.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:16:00.518 INFO:tasks.workunit.client.0.trial111.stdout:osd.5 up in weight 1 up_from 44 up_thru 45 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/520486221,v1:127.0.0.1:6843/520486221] [v2:127.0.0.1:6844/520486221,v1:127.0.0.1:6845/520486221] exists,up f0d17a3f-6114-4920-8826-310a8d56cf85 2026-02-19T13:16:00.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:00.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:00.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:00.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:16:00.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T13:16:00.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:16:00.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:00.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:16:00.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:16:00.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=d7e15df8-25fa-4c54-9e2a-929230d847f4 2026-02-19T13:16:00.526 INFO:tasks.workunit.client.0.trial111.stdout:add osd6 d7e15df8-25fa-4c54-9e2a-929230d847f4 2026-02-19T13:16:00.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 d7e15df8-25fa-4c54-9e2a-929230d847f4' 2026-02-19T13:16:00.526 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:16:00.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAQDZdp0EjfHxAAjOG9WtoozTTBWNSdimGBBg== 2026-02-19T13:16:00.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAQDZdp0EjfHxAAjOG9WtoozTTBWNSdimGBBg=="}' 2026-02-19T13:16:00.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new d7e15df8-25fa-4c54-9e2a-929230d847f4 -i td/test-erasure-eio/6/new.json 2026-02-19T13:16:00.754 INFO:tasks.workunit.client.0.trial111.stdout:6 2026-02-19T13:16:00.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T13:16:00.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAQDZdp0EjfHxAAjOG9WtoozTTBWNSdimGBBg== --osd-uuid d7e15df8-25fa-4c54-9e2a-929230d847f4 2026-02-19T13:16:00.783 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:00.784+0000 7f0291451900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:00.785 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:00.785+0000 7f0291451900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:00.785 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:00.786+0000 7f0291451900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:00.786 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:00.786+0000 7f0291451900 -1 bdev(0x55e245c84c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T13:16:00.786 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:00.786+0000 7f0291451900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T13:16:01.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T13:16:01.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:01.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T13:16:01.220 INFO:tasks.workunit.client.0.trial111.stdout:adding osd6 key to auth repository 2026-02-19T13:16:01.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:01.521 INFO:tasks.workunit.client.0.trial111.stdout:start osd.6 2026-02-19T13:16:01.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T13:16:01.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:01.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:01.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:01.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:01.537 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:01.537+0000 7faa520b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:01.538 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:01.539+0000 7faa520b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:01.539 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:01.539+0000 7faa520b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:01.727 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:01.727+0000 7faa520b6900 -1 Falling back to public interface 2026-02-19T13:16:01.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T13:16:01.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:01.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T13:16:01.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:01.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:01.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:01.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:01.744 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:16:01.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:01.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:16:01.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:02.100 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:02.100+0000 7faa520b6900 -1 osd.6 0 log_to_monitors true 2026-02-19T13:16:02.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:02.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:02.971 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:02.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:02.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:02.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:16:03.194 INFO:tasks.workunit.client.0.trial111.stdout:osd.6 up in weight 1 up_from 52 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/2767146881,v1:127.0.0.1:6851/2767146881] [v2:127.0.0.1:6852/2767146881,v1:127.0.0.1:6853/2767146881] exists,up d7e15df8-25fa-4c54-9e2a-929230d847f4 2026-02-19T13:16:03.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:03.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:03.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:16:03.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:16:03.257 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:16:03.264 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:16:03.267 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:15:45.647+0000 7f0ea186e900 0 load: jerasure load: lrc 2026-02-19T13:16:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:506: TEST_ec_recovery_multiple_objects_eio: CEPH_ARGS='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:16:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:508: TEST_ec_recovery_multiple_objects_eio: local poolname=pool-jerasure 2026-02-19T13:16:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:509: TEST_ec_recovery_multiple_objects_eio: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T13:16:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:16:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:16:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T13:16:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:16:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T13:16:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:16:03.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T13:16:03.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:16:03.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:16:03.909 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:16:03.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:16:04.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:16:04.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:16:04.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:16:04.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:16:04.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:16:04.922 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:16:04.923 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:16:04.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:16:04.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:16:04.923 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:16:05.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:16:05.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:16:05.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:16:05.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:16:05.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:16:05.002 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:16:05.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:16:05.222 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:16:05.222 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:16:05.222 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:16:05.222 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:16:05.222 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:16:05.222 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:16:05.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:16:05.223 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:05.223 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:16:05.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-19T13:16:05.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-19T13:16:05.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782' 2026-02-19T13:16:05.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:05.302 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:16:05.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607558 2026-02-19T13:16:05.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607558 2026-02-19T13:16:05.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558' 2026-02-19T13:16:05.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:05.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:16:05.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411333 2026-02-19T13:16:05.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411333 2026-02-19T13:16:05.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333' 2026-02-19T13:16:05.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:05.461 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:16:05.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149701 2026-02-19T13:16:05.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149701 2026-02-19T13:16:05.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701' 2026-02-19T13:16:05.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:05.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:16:05.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-19T13:16:05.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-19T13:16:05.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-150323855364' 2026-02-19T13:16:05.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:05.622 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:16:05.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561028 2026-02-19T13:16:05.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561028 2026-02-19T13:16:05.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-150323855364 5-188978561028' 2026-02-19T13:16:05.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:05.699 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:16:05.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299395 2026-02-19T13:16:05.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299395 2026-02-19T13:16:05.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607558 2-77309411333 3-111669149701 4-150323855364 5-188978561028 6-223338299395' 2026-02-19T13:16:05.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:05.775 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-19T13:16:05.776 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:05.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:16:05.777 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803782 2026-02-19T13:16:05.777 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:05.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803782 2026-02-19T13:16:05.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-19T13:16:05.779 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803782 2026-02-19T13:16:05.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:05.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-19T13:16:05.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:06.000 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607558 2026-02-19T13:16:06.000 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:06.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:16:06.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607558 2026-02-19T13:16:06.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:06.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607558 2026-02-19T13:16:06.003 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607558' 2026-02-19T13:16:06.003 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 51539607558 2026-02-19T13:16:06.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:16:06.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607558 -lt 51539607558 2026-02-19T13:16:06.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:06.226 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411333 2026-02-19T13:16:06.226 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:06.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:16:06.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411333 2026-02-19T13:16:06.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:06.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411333 2026-02-19T13:16:06.229 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411333' 2026-02-19T13:16:06.229 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411333 2026-02-19T13:16:06.229 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:16:06.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411333 -lt 77309411333 2026-02-19T13:16:06.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:06.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149701 2026-02-19T13:16:06.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:06.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:16:06.453 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149701 2026-02-19T13:16:06.453 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:06.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149701 2026-02-19T13:16:06.454 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149701' 2026-02-19T13:16:06.454 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149701 2026-02-19T13:16:06.454 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:16:06.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149701 -lt 111669149701 2026-02-19T13:16:06.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:06.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855364 2026-02-19T13:16:06.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:06.679 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:16:06.680 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855364 2026-02-19T13:16:06.680 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:06.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855364 2026-02-19T13:16:06.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855364' 2026-02-19T13:16:06.681 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 150323855364 2026-02-19T13:16:06.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:16:06.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-19T13:16:06.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:06.903 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561028 2026-02-19T13:16:06.903 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:06.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:16:06.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561028 2026-02-19T13:16:06.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:06.906 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561028 2026-02-19T13:16:06.906 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561028' 2026-02-19T13:16:06.906 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 188978561028 2026-02-19T13:16:06.906 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:16:07.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561028 -lt 188978561028 2026-02-19T13:16:07.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:07.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299395 2026-02-19T13:16:07.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:07.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:16:07.128 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299395 2026-02-19T13:16:07.128 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:07.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299395 2026-02-19T13:16:07.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299395' 2026-02-19T13:16:07.129 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 223338299395 2026-02-19T13:16:07.130 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:16:07.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299394 -lt 223338299395 2026-02-19T13:16:07.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:16:08.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:16:08.352 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:16:08.573 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299395 -lt 223338299395 2026-02-19T13:16:08.573 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:16:08.574 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:08.574 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:08.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:16:08.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:16:08.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:16:08.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:16:08.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:16:08.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:16:08.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:16:08.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:16:09.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:16:09.088 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:16:09.088 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:09.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:09.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:16:09.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:16:09.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:16:09.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:511: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test1 2026-02-19T13:16:09.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test1 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:16:09.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test1 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:09.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:512: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test2 2026-02-19T13:16:09.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:16:09.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test2 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:16:09.409 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test2 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:513: TEST_ec_recovery_multiple_objects_eio: rados_put td/test-erasure-eio pool-jerasure test3 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=test3 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:16:09.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:16:09.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:09.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:16:09.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put test3 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:09.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:516: TEST_ec_recovery_multiple_objects_eio: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-19T13:16:09.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:16:09.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:16:09.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:16:09.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:16:09.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:16:09.445 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-19T13:16:09.665 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:16:09.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-19T13:16:09.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:16:09.666 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T13:16:09.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:16:09.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T13:16:09.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:16:09.666 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T13:16:09.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:16:09.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:16:09.667 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:16:09.667 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T13:16:09.667 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:16:09.667 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:16:09.668 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:09.668 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:09.668 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:09.668 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:16:09.668 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:16:09.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:16:09.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:16:09.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-19T13:16:09.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:517: TEST_ec_recovery_multiple_objects_eio: ceph osd out 0 2026-02-19T13:16:10.069 INFO:tasks.workunit.client.0.trial111.stderr:osd.0 is already out. 2026-02-19T13:16:10.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:520: TEST_ec_recovery_multiple_objects_eio: wait_for_clean 2026-02-19T13:16:10.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:16:10.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:16:10.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:16:10.082 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:16:10.083 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:16:10.083 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:16:10.083 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:16:10.083 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:16:10.083 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:16:10.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:16:10.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:16:10.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:16:10.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:16:10.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:16:10.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:16:10.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:16:10.386 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:16:10.386 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:16:10.386 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:16:10.387 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:16:10.387 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:16:10.387 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:16:10.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:16:10.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:10.387 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:16:10.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-19T13:16:10.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-19T13:16:10.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785' 2026-02-19T13:16:10.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:10.469 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:16:10.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607561 2026-02-19T13:16:10.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607561 2026-02-19T13:16:10.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607561' 2026-02-19T13:16:10.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:10.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:16:10.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411336 2026-02-19T13:16:10.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411336 2026-02-19T13:16:10.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336' 2026-02-19T13:16:10.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:10.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:16:10.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149704 2026-02-19T13:16:10.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149704 2026-02-19T13:16:10.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-111669149704' 2026-02-19T13:16:10.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:10.706 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:16:10.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855367 2026-02-19T13:16:10.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855367 2026-02-19T13:16:10.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-111669149704 4-150323855367' 2026-02-19T13:16:10.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:10.784 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:16:10.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=188978561031 2026-02-19T13:16:10.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 188978561031 2026-02-19T13:16:10.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-111669149704 4-150323855367 5-188978561031' 2026-02-19T13:16:10.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:10.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:16:10.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=223338299398 2026-02-19T13:16:10.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 223338299398 2026-02-19T13:16:10.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607561 2-77309411336 3-111669149704 4-150323855367 5-188978561031 6-223338299398' 2026-02-19T13:16:10.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:10.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-19T13:16:10.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:10.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:16:10.944 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803785 2026-02-19T13:16:10.944 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:10.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803785 2026-02-19T13:16:10.946 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803785 2026-02-19T13:16:10.946 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-19T13:16:10.946 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:11.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-19T13:16:11.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:16:12.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:16:12.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:12.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-19T13:16:12.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:12.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607561 2026-02-19T13:16:12.390 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:12.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:16:12.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607561 2026-02-19T13:16:12.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:12.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607561 2026-02-19T13:16:12.392 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 51539607561 2026-02-19T13:16:12.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607561' 2026-02-19T13:16:12.393 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:16:12.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607561 -lt 51539607561 2026-02-19T13:16:12.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:12.619 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411336 2026-02-19T13:16:12.619 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:12.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:16:12.621 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411336 2026-02-19T13:16:12.621 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:12.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411336 2026-02-19T13:16:12.622 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411336 2026-02-19T13:16:12.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411336' 2026-02-19T13:16:12.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:16:12.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411336 -lt 77309411336 2026-02-19T13:16:12.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:12.847 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149704 2026-02-19T13:16:12.847 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:12.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:16:12.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149704 2026-02-19T13:16:12.848 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:12.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149704 2026-02-19T13:16:12.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149704' 2026-02-19T13:16:12.849 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149704 2026-02-19T13:16:12.850 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:16:13.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149704 -lt 111669149704 2026-02-19T13:16:13.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:13.072 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855367 2026-02-19T13:16:13.072 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:13.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:16:13.073 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855367 2026-02-19T13:16:13.073 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:13.074 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855367 2026-02-19T13:16:13.074 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 150323855367 2026-02-19T13:16:13.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855367' 2026-02-19T13:16:13.075 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:16:13.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855367 -lt 150323855367 2026-02-19T13:16:13.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:13.296 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-188978561031 2026-02-19T13:16:13.296 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:13.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:16:13.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-188978561031 2026-02-19T13:16:13.298 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:13.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=188978561031 2026-02-19T13:16:13.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 188978561031' 2026-02-19T13:16:13.299 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 188978561031 2026-02-19T13:16:13.299 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:16:13.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 188978561031 -lt 188978561031 2026-02-19T13:16:13.522 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:13.522 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-223338299398 2026-02-19T13:16:13.522 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:13.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:16:13.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-223338299398 2026-02-19T13:16:13.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:13.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=223338299398 2026-02-19T13:16:13.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 223338299398' 2026-02-19T13:16:13.525 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 223338299398 2026-02-19T13:16:13.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:16:13.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 223338299398 -lt 223338299398 2026-02-19T13:16:13.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:16:13.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:13.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:14.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:16:14.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:16:14.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:16:14.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:16:14.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:16:14.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:16:14.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:16:14.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:16:14.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:16:14.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:16:14.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:14.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:14.562 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:522: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test1 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test1 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:16:14.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test1 td/test-erasure-eio/COPY 2026-02-19T13:16:14.580 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:16:14.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:16:14.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:523: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test2 2026-02-19T13:16:14.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:16:14.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:16:14.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test2 2026-02-19T13:16:14.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:16:14.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:16:14.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test2 td/test-erasure-eio/COPY 2026-02-19T13:16:14.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:16:14.600 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:16:14.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:524: TEST_ec_recovery_multiple_objects_eio: rados_get td/test-erasure-eio pool-jerasure test3 2026-02-19T13:16:14.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:16:14.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:16:14.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=test3 2026-02-19T13:16:14.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:16:14.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:16:14.602 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get test3 td/test-erasure-eio/COPY 2026-02-19T13:16:14.617 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:16:14.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:16:14.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:526: TEST_ec_recovery_multiple_objects_eio: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:16:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:16:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:16:14.880 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:16:14.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:16:15.160 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:16:15.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:16:15.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:16:15.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:16:15.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:16:15.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:16:15.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:16:15.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:16:15.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:16:15.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:16:15.280 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:16:15.280 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:16:15.281 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:16:15.281 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:16:15.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:16:15.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:16:15.283 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:16:15.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:16:15.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:16:15.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:16:15.284 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:16:15.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:16:15.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:16:15.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:16:15.299 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:16:15.300 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:15.300 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:15.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:16:15.302 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:16:15.303 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:16:15.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:16:15.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:16:15.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:16:15.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:16:15.303 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:16:15.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:16:15.304 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:16:15.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:16:15.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:16:15.306 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:16:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:16:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:16:15.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:16:15.307 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:16:15.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:16:15.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:16:15.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:16:15.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:16:15.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:15.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:15.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:16:15.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:16:15.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:16:15.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:16:15.313 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:16:15.313 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:15.313 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:15.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:16:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:16:15.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:16:15.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:16:15.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:16:15.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:15.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:15.335 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:15.335 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:15.335 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:15.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:15.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:16:15.358 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:16:15.359 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:16:15.359 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:16:15.359 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:16:15.360 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:16:15.360 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:16:15.361 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:16:15.361 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:16:15.361 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:16:15.361 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:15.361 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:15.361 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:15.361 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:16:15.362 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:16:15.362 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:16:15.424 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:15.425 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:15.426 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:16:15.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:16:15.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:16:15.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:16:15.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:16:15.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:16:15.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:16:15.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:16:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:16:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:16:15.595 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:16:15.595 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:15.595 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:15.595 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:15.596 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:15.596 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:15.596 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:15.596 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:16:15.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:16:15.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:16:15.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:16:15.714 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:16:15.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:16:16.725 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:16:16.725 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:16:16.725 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:16:16.726 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:16.726 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:16.726 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:16.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:16:16.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:16:16.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:16:16.780 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:16:16.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:16:16.788 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:16:15.352+0000 7fa3ba13cdc0 0 load: jerasure load: lrc 2026-02-19T13:16:16.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_recovery_unfound td/test-erasure-eio 2026-02-19T13:16:16.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:629: TEST_ec_recovery_unfound: local dir=td/test-erasure-eio 2026-02-19T13:16:16.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:630: TEST_ec_recovery_unfound: local objname=myobject 2026-02-19T13:16:16.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:631: TEST_ec_recovery_unfound: local lastobj=100 2026-02-19T13:16:16.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:633: TEST_ec_recovery_unfound: local testobj=obj75 2026-02-19T13:16:16.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:635: TEST_ec_recovery_unfound: ORIG_ARGS='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:16:16.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:636: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 ' 2026-02-19T13:16:16.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:637: TEST_ec_recovery_unfound: CEPH_ARGS+=' --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:16:16.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:638: TEST_ec_recovery_unfound: setup_osds 5 2026-02-19T13:16:16.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=5 2026-02-19T13:16:16.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:16:16.790 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 5 - 1 2026-02-19T13:16:16.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 4 2026-02-19T13:16:16.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:16:16.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:16:16.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:16:16.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:16:16.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:16:16.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:16:16.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:16.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=8f3556e4-2328-4f48-bf0a-842b3c5a3c43 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 8f3556e4-2328-4f48-bf0a-842b3c5a3c43' 2026-02-19T13:16:16.797 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 8f3556e4-2328-4f48-bf0a-842b3c5a3c43 2026-02-19T13:16:16.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:16:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAgDZdpSsokMBAADyOCs4wKxXurcLVF7z+BTg== 2026-02-19T13:16:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAgDZdpSsokMBAADyOCs4wKxXurcLVF7z+BTg=="}' 2026-02-19T13:16:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 8f3556e4-2328-4f48-bf0a-842b3c5a3c43 -i td/test-erasure-eio/0/new.json 2026-02-19T13:16:17.030 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:16:17.042 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:16:17.043 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAgDZdpSsokMBAADyOCs4wKxXurcLVF7z+BTg== --osd-uuid 8f3556e4-2328-4f48-bf0a-842b3c5a3c43 2026-02-19T13:16:17.061 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.061+0000 7f4d9d39c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:17.063 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.063+0000 7f4d9d39c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:17.063 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.064+0000 7f4d9d39c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:17.064 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.064+0000 7f4d9d39c900 -1 bdev(0x561ef8716c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:16:17.064 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.064+0000 7f4d9d39c900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:16:17.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:16:17.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:17.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:16:17.450 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:16:17.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:17.752 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:16:17.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:16:17.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:17.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:17.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:17.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:17.770 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.770+0000 7fa21b0a6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:17.771 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.771+0000 7fa21b0a6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:17.772 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.772+0000 7fa21b0a6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:17.974 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:16:17.974 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:17.974 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:16:17.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:17.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:17.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:17.975 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:16:17.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:17.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:17.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:16:17.980 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:17.981+0000 7fa21b0a6900 -1 Falling back to public interface 2026-02-19T13:16:18.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:18.372 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:18.372+0000 7fa21b0a6900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:16:19.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:19.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:19.199 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:19.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:19.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:19.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:16:19.426 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1709061993,v1:127.0.0.1:6803/1709061993] [v2:127.0.0.1:6804/1709061993,v1:127.0.0.1:6805/1709061993] exists,up 8f3556e4-2328-4f48-bf0a-842b3c5a3c43 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:16:19.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:19.432 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b38f6e24-c2f3-403c-9a22-90ba828a1702 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b38f6e24-c2f3-403c-9a22-90ba828a1702' 2026-02-19T13:16:19.434 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 b38f6e24-c2f3-403c-9a22-90ba828a1702 2026-02-19T13:16:19.435 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:16:19.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAjDZdpQNB2GhAAtpZteuSC5uYVf5tvmF8XkQ== 2026-02-19T13:16:19.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAjDZdpQNB2GhAAtpZteuSC5uYVf5tvmF8XkQ=="}' 2026-02-19T13:16:19.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b38f6e24-c2f3-403c-9a22-90ba828a1702 -i td/test-erasure-eio/1/new.json 2026-02-19T13:16:19.666 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:19.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:16:19.679 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAjDZdpQNB2GhAAtpZteuSC5uYVf5tvmF8XkQ== --osd-uuid b38f6e24-c2f3-403c-9a22-90ba828a1702 2026-02-19T13:16:19.697 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:19.697+0000 7fa3f08fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:19.698 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:19.698+0000 7fa3f08fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:19.699 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:19.699+0000 7fa3f08fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:19.700 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:19.700+0000 7fa3f08fe900 -1 bdev(0x55c2b63c4c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:16:19.700 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:19.700+0000 7fa3f08fe900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:16:20.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:16:20.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:20.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:16:20.132 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:16:20.133 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:20.433 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:16:20.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:16:20.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:20.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:20.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:20.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:20.450 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:20.450+0000 7f6153840900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:20.452 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:20.452+0000 7f6153840900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:20.452 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:20.453+0000 7f6153840900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:20.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:16:20.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:20.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:16:20.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:20.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:20.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:20.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:20.657 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:16:20.658 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:20.658 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:20.658 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:20.656+0000 7f6153840900 -1 Falling back to public interface 2026-02-19T13:16:20.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:21.055 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:21.055+0000 7f6153840900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:16:21.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:21.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:21.877 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:21.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:21.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:21.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:16:22.103 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1709216071,v1:127.0.0.1:6811/1709216071] [v2:127.0.0.1:6812/1709216071,v1:127.0.0.1:6813/1709216071] exists,up b38f6e24-c2f3-403c-9a22-90ba828a1702 2026-02-19T13:16:22.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:22.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:22.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:22.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:22.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:16:22.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=155d6236-32ae-45b1-8ca8-5cb9a8af7b2b 2026-02-19T13:16:22.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 155d6236-32ae-45b1-8ca8-5cb9a8af7b2b' 2026-02-19T13:16:22.111 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 155d6236-32ae-45b1-8ca8-5cb9a8af7b2b 2026-02-19T13:16:22.111 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:16:22.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAmDZdpQns0BxAAIPCITNT60VhGC5vKWlMnEw== 2026-02-19T13:16:22.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAmDZdpQns0BxAAIPCITNT60VhGC5vKWlMnEw=="}' 2026-02-19T13:16:22.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 155d6236-32ae-45b1-8ca8-5cb9a8af7b2b -i td/test-erasure-eio/2/new.json 2026-02-19T13:16:22.344 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:16:22.355 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:16:22.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAmDZdpQns0BxAAIPCITNT60VhGC5vKWlMnEw== --osd-uuid 155d6236-32ae-45b1-8ca8-5cb9a8af7b2b 2026-02-19T13:16:22.374 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:22.374+0000 7fbf59344900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:22.375 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:22.376+0000 7fbf59344900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:22.376 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:22.377+0000 7fbf59344900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:22.377 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:22.377+0000 7fbf59344900 -1 bdev(0x55c77c3dac00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:16:22.377 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:22.377+0000 7fbf59344900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:16:22.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:16:22.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:22.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:16:22.810 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:16:22.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:23.115 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:16:23.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:16:23.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:23.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:23.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:23.116 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:23.132 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:23.132+0000 7f84391e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:23.133 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:23.133+0000 7f84391e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:23.134 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:23.134+0000 7f84391e0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:23.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:16:23.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:23.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:16:23.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:23.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:23.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:23.340 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:16:23.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:23.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:23.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:16:23.345 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:23.345+0000 7f84391e0900 -1 Falling back to public interface 2026-02-19T13:16:23.561 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:23.658 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:23.658+0000 7f84391e0900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:16:24.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:24.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:24.563 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:24.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:24.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:24.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:16:24.790 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2986491759,v1:127.0.0.1:6819/2986491759] [v2:127.0.0.1:6820/2986491759,v1:127.0.0.1:6821/2986491759] exists,up 155d6236-32ae-45b1-8ca8-5cb9a8af7b2b 2026-02-19T13:16:24.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:24.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:16:24.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:24.796 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:24.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:24.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:16:24.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:16:24.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:16:24.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:16:24.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=62fa97fc-800f-4158-ba4f-1306b1ed0529 2026-02-19T13:16:24.798 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 62fa97fc-800f-4158-ba4f-1306b1ed0529 2026-02-19T13:16:24.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 62fa97fc-800f-4158-ba4f-1306b1ed0529' 2026-02-19T13:16:24.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:16:24.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAoDZdpvdAoMBAAmgFwVm6Z/xRUryfMbCcy/A== 2026-02-19T13:16:24.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAoDZdpvdAoMBAAmgFwVm6Z/xRUryfMbCcy/A=="}' 2026-02-19T13:16:24.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 62fa97fc-800f-4158-ba4f-1306b1ed0529 -i td/test-erasure-eio/3/new.json 2026-02-19T13:16:25.029 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:16:25.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:16:25.041 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAoDZdpvdAoMBAAmgFwVm6Z/xRUryfMbCcy/A== --osd-uuid 62fa97fc-800f-4158-ba4f-1306b1ed0529 2026-02-19T13:16:25.060 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:25.060+0000 7f2f24c9c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:25.061 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:25.061+0000 7f2f24c9c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:25.062 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:25.062+0000 7f2f24c9c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:25.062 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:25.062+0000 7f2f24c9c900 -1 bdev(0x55d75fcf8c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:16:25.062 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:25.062+0000 7f2f24c9c900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:16:25.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:16:25.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:25.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:16:25.487 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:16:25.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:25.791 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:16:25.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:16:25.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:25.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:25.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:25.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:25.808 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:25.808+0000 7efc72d30900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:25.809 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:25.810+0000 7efc72d30900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:25.810 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:25.811+0000 7efc72d30900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:26.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:16:26.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:26.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:16:26.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:26.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:26.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:26.018 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:16:26.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:26.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:26.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:16:26.039 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:26.040+0000 7efc72d30900 -1 Falling back to public interface 2026-02-19T13:16:26.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:26.432 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:26.432+0000 7efc72d30900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:16:27.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:27.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:27.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:27.242 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:27.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:27.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:16:27.466 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4080918687,v1:127.0.0.1:6827/4080918687] [v2:127.0.0.1:6828/4080918687,v1:127.0.0.1:6829/4080918687] exists,up 62fa97fc-800f-4158-ba4f-1306b1ed0529 2026-02-19T13:16:27.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:16:27.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10' 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:16:27.472 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:16:27.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=f584724f-7202-4e7a-bf7c-ef753cf823a4 2026-02-19T13:16:27.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 f584724f-7202-4e7a-bf7c-ef753cf823a4' 2026-02-19T13:16:27.475 INFO:tasks.workunit.client.0.trial111.stdout:add osd4 f584724f-7202-4e7a-bf7c-ef753cf823a4 2026-02-19T13:16:27.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:16:27.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQArDZdpMlrjHBAAfnZ3jx8FfWM+wEYYVBK0Pw== 2026-02-19T13:16:27.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQArDZdpMlrjHBAAfnZ3jx8FfWM+wEYYVBK0Pw=="}' 2026-02-19T13:16:27.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new f584724f-7202-4e7a-bf7c-ef753cf823a4 -i td/test-erasure-eio/4/new.json 2026-02-19T13:16:27.706 INFO:tasks.workunit.client.0.trial111.stdout:4 2026-02-19T13:16:27.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T13:16:27.719 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQArDZdpMlrjHBAAfnZ3jx8FfWM+wEYYVBK0Pw== --osd-uuid f584724f-7202-4e7a-bf7c-ef753cf823a4 2026-02-19T13:16:27.737 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:27.738+0000 7f509f556900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:27.738 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:27.740+0000 7f509f556900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:27.739 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:27.741+0000 7f509f556900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:27.740 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:27.741+0000 7f509f556900 -1 bdev(0x558b0544cc00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:16:27.740 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:27.741+0000 7f509f556900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T13:16:28.157 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T13:16:28.157 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:16:28.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T13:16:28.158 INFO:tasks.workunit.client.0.trial111.stdout:adding osd4 key to auth repository 2026-02-19T13:16:28.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:16:28.458 INFO:tasks.workunit.client.0.trial111.stdout:start osd.4 2026-02-19T13:16:28.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T13:16:28.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-recovery-max-single-start 3 --osd-recovery-max-active 3 --osd_min_pg_log_entries=5 --osd_max_pg_log_entries=10 --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:28.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:16:28.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:16:28.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:16:28.476 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:28.477+0000 7f9643389900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:28.477 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:28.478+0000 7f9643389900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:28.478 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:28.479+0000 7f9643389900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:28.665 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:28.666+0000 7f9643389900 -1 Falling back to public interface 2026-02-19T13:16:28.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T13:16:28.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:28.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T13:16:28.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:28.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:28.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:28.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:28.683 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:16:28.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:28.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:16:28.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:29.041 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:29.042+0000 7f9643389900 -1 osd.4 0 log_to_monitors true 2026-02-19T13:16:29.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:29.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:29.903 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:29.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:29.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:29.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:16:30.125 INFO:tasks.workunit.client.0.trial111.stdout:osd.4 up in weight 1 up_from 34 up_thru 35 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/23211558,v1:127.0.0.1:6835/23211558] [v2:127.0.0.1:6836/23211558,v1:127.0.0.1:6837/23211558] exists,up f584724f-7202-4e7a-bf7c-ef753cf823a4 2026-02-19T13:16:30.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:30.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:30.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:30.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:16:30.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:16:30.128 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:16:30.128 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:30.128 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:30.128 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:30.128 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:16:30.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:16:30.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:16:30.187 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:16:30.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:16:30.196 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:16:17.983+0000 7fa21b0a6900 0 load: jerasure load: lrc 2026-02-19T13:16:30.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:639: TEST_ec_recovery_unfound: CEPH_ARGS='--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:16:30.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:641: TEST_ec_recovery_unfound: local poolname=pool-jerasure 2026-02-19T13:16:30.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:642: TEST_ec_recovery_unfound: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T13:16:30.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:16:30.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:16:30.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T13:16:30.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:16:30.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T13:16:30.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:16:30.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T13:16:30.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:16:30.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:16:30.848 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:16:30.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:16:31.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:16:31.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:16:31.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:16:31.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:16:31.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:16:31.864 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:16:31.864 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:16:31.864 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:16:31.864 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:16:31.864 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:16:31.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:16:31.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:16:31.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:16:31.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:16:31.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:16:31.942 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:16:32.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:16:32.162 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:16:32.162 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:16:32.162 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:16:32.162 INFO:tasks.workunit.client.0.trial111.stderr:4' 2026-02-19T13:16:32.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:16:32.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:32.162 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:16:32.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T13:16:32.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T13:16:32.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T13:16:32.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:32.242 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:16:32.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-19T13:16:32.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-19T13:16:32.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-19T13:16:32.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:32.322 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:16:32.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378628 2026-02-19T13:16:32.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378628 2026-02-19T13:16:32.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628' 2026-02-19T13:16:32.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:32.401 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:16:32.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T13:16:32.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T13:16:32.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628 3-111669149699' 2026-02-19T13:16:32.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:16:32.477 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:16:32.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888067 2026-02-19T13:16:32.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888067 2026-02-19T13:16:32.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-81604378628 3-111669149699 4-146028888067' 2026-02-19T13:16:32.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:32.559 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T13:16:32.559 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:32.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:16:32.560 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-19T13:16:32.561 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:32.561 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-19T13:16:32.561 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T13:16:32.562 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803781 2026-02-19T13:16:32.562 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:32.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-19T13:16:32.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:16:33.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:16:33.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:34.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803779 -lt 25769803781 2026-02-19T13:16:34.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:16:35.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:16:35.008 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:16:35.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-19T13:16:35.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:35.234 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-19T13:16:35.234 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:35.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:16:35.235 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574853 2026-02-19T13:16:35.235 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:35.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574853 2026-02-19T13:16:35.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-19T13:16:35.237 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574853 2026-02-19T13:16:35.237 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:16:35.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-19T13:16:35.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:35.459 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378628 2026-02-19T13:16:35.460 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:35.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:16:35.460 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-81604378628 2026-02-19T13:16:35.460 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:35.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378628 2026-02-19T13:16:35.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 81604378628' 2026-02-19T13:16:35.462 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 81604378628 2026-02-19T13:16:35.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:16:35.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378628 -lt 81604378628 2026-02-19T13:16:35.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:35.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T13:16:35.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:35.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:16:35.683 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149699 2026-02-19T13:16:35.683 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:35.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149699 2026-02-19T13:16:35.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T13:16:35.685 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149699 2026-02-19T13:16:35.685 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:16:35.906 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149700 -lt 111669149699 2026-02-19T13:16:35.906 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:16:35.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888067 2026-02-19T13:16:35.907 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:16:35.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:16:35.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888067 2026-02-19T13:16:35.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:16:35.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888067 2026-02-19T13:16:35.910 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 146028888067 2026-02-19T13:16:35.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888067' 2026-02-19T13:16:35.910 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:16:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888068 -lt 146028888067 2026-02-19T13:16:36.129 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:16:36.129 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:36.129 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:36.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:16:36.426 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:16:36.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:16:36.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:16:36.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:16:36.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:16:36.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:16:36.426 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:16:36.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:16:36.648 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:16:36.648 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:16:36.648 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:16:36.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:16:36.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:16:36.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:16:36.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:644: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-19T13:16:37.152 INFO:tasks.workunit.client.0.trial111.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:16:37.152 INFO:tasks.workunit.client.0.trial111.stdout:2.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:31.460575+0000 0'0 41:16 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T13:16:30.792889+0000 0'0 2026-02-19T13:16:30.792889+0000 0 0 periodic scrub scheduled @ 2026-02-20T14:44:48.015436+0000 0 0 2026-02-19T13:16:37.152 INFO:tasks.workunit.client.0.trial111.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:30.412942+0000 0'0 41:21 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T21:30:19.458062+0000 0 0 2026-02-19T13:16:37.152 INFO:tasks.workunit.client.0.trial111.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:30.118508+0000 0'0 41:97 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T14:12:22.424617+0000 0 0 2026-02-19T13:16:37.152 INFO:tasks.workunit.client.0.trial111.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:30.419946+0000 0'0 41:21 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T17:04:49.581876+0000 0 0 2026-02-19T13:16:37.153 INFO:tasks.workunit.client.0.trial111.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:24.373919+0000 0'0 41:63 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T16:59:18.337505+0000 0 0 2026-02-19T13:16:37.153 INFO:tasks.workunit.client.0.trial111.stdout: 2026-02-19T13:16:37.153 INFO:tasks.workunit.client.0.trial111.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:16:37.153 INFO:tasks.workunit.client.0.trial111.stderr:dumped pgs 2026-02-19T13:16:37.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:646: TEST_ec_recovery_unfound: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T13:16:37.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:16:37.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:16:37.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T13:16:37.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:37.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:16:37.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:37.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:16:37.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:37.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:16:37.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:16:37.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:16:37.165 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:37.182 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: get_osds pool-jerasure myobject 2026-02-19T13:16:37.182 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:16:37.182 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:16:37.183 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:16:37.183 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:16:37.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:16:37.405 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:16:37.405 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:16:37.405 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: initial_osds=('3' '1' '4' '0' '2') 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:648: TEST_ec_recovery_unfound: local -a initial_osds 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:649: TEST_ec_recovery_unfound: local last_osd=2 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:650: TEST_ec_recovery_unfound: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:16:37.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:16:37.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:16:37.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:651: TEST_ec_recovery_unfound: ceph osd down 2 2026-02-19T13:16:37.926 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already down. 2026-02-19T13:16:37.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:652: TEST_ec_recovery_unfound: ceph osd out 2 2026-02-19T13:16:38.180 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already out. 2026-02-19T13:16:38.192 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:654: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-19T13:16:38.399 INFO:tasks.workunit.client.0.trial111.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:16:38.400 INFO:tasks.workunit.client.0.trial111.stdout:2.0 1 0 0 0 0 4096 0 0 1 0 1 peering 2026-02-19T13:16:37.492317+0000 41'1 41:18 [3,1,4,0,NONE] 3 [3,1,4,0,NONE] 3 0'0 2026-02-19T13:16:30.792889+0000 0'0 2026-02-19T13:16:30.792889+0000 0 0 no scrub is scheduled 0 0 2026-02-19T13:16:38.400 INFO:tasks.workunit.client.0.trial111.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:30.412942+0000 0'0 41:21 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T21:30:19.458062+0000 0 0 2026-02-19T13:16:38.400 INFO:tasks.workunit.client.0.trial111.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:30.118508+0000 0'0 41:97 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T14:12:22.424617+0000 0 0 2026-02-19T13:16:38.400 INFO:tasks.workunit.client.0.trial111.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:30.419946+0000 0'0 41:21 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T17:04:49.581876+0000 0 0 2026-02-19T13:16:38.401 INFO:tasks.workunit.client.0.trial111.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:24.373919+0000 0'0 41:63 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T16:59:18.337505+0000 0 0 2026-02-19T13:16:38.401 INFO:tasks.workunit.client.0.trial111.stdout: 2026-02-19T13:16:38.401 INFO:tasks.workunit.client.0.trial111.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:16:38.401 INFO:tasks.workunit.client.0.trial111.stderr:dumped pgs 2026-02-19T13:16:38.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:656: TEST_ec_recovery_unfound: dd if=/dev/urandom of=td/test-erasure-eio/ORIGINAL bs=1024 count=4 2026-02-19T13:16:38.412 INFO:tasks.workunit.client.0.trial111.stderr:4+0 records in 2026-02-19T13:16:38.412 INFO:tasks.workunit.client.0.trial111.stderr:4+0 records out 2026-02-19T13:16:38.412 INFO:tasks.workunit.client.0.trial111.stderr:4096 bytes (4.1 kB, 4.0 KiB) copied, 5.1906e-05 s, 78.9 MB/s 2026-02-19T13:16:38.413 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: seq 1 100 2026-02-19T13:16:38.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj1 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj2 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj3 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj4 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj5 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj6 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj7 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj8 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.560 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj9 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.578 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.578 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj10 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.595 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.595 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj11 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj12 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj13 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj14 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj15 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.685 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.685 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj16 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj17 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.720 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.720 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj18 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj19 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj20 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj21 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj22 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj23 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.827 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.827 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj24 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.845 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj25 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj26 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj27 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.898 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.898 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj28 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj29 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj30 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj31 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj32 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:38.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:38.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj33 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj34 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.022 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.022 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj35 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj36 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj37 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.076 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.076 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj38 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj39 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.112 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj40 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj41 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj42 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj43 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj44 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj45 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj46 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj47 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj48 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj49 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj50 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj51 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj52 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj53 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj54 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.380 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj55 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj56 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj57 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj58 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj59 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj60 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj61 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj62 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj63 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj64 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj65 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj66 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj67 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj68 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.626 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.626 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj69 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj70 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj71 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj72 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.698 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.698 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj73 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj74 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj75 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj76 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj77 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj78 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.804 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.804 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj79 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj80 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj81 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj82 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj83 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj84 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj85 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj86 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj87 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj88 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:39.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:39.984 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj89 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj90 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj91 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj92 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj93 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj94 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj95 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj96 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj97 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj98 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj99 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:657: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:16:40.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:659: TEST_ec_recovery_unfound: rados --pool pool-jerasure put obj100 td/test-erasure-eio/ORIGINAL 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:662: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 0 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj75 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:16:40.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:16:40.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj75 2026-02-19T13:16:40.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:16:40.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-19T13:16:40.194 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-19T13:16:40.194 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:16:40.420 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr:2147483647' 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-19T13:16:40.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:16:40.422 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T13:16:40.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:40.424 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:40.425 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:16:40.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:16:40.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:16:40.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:16:40.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:16:40.493 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-19T13:16:40.493 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:16:40.493 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:16:40.493 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:40.493 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:40.493 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:40.493 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:16:40.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:16:40.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.3.asok injectdataerr pool-jerasure obj75 0 2026-02-19T13:16:40.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:663: TEST_ec_recovery_unfound: inject_eio ec data pool-jerasure obj75 td/test-erasure-eio 1 2026-02-19T13:16:40.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:16:40.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:16:40.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj75 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj75 2026-02-19T13:16:40.559 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:16:40.560 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj75 2026-02-19T13:16:40.560 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj75 2026-02-19T13:16:40.560 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:16:40.781 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:16:40.781 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:16:40.781 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:16:40.781 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:16:40.781 INFO:tasks.workunit.client.0.trial111.stderr:2147483647' 2026-02-19T13:16:40.781 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 4 0 2147483647 2026-02-19T13:16:40.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '4' '0' '2147483647') 2026-02-19T13:16:40.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:16:40.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:16:40.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:16:40.782 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:16:40.784 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:16:40.785 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:40.785 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:40.785 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:40.785 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:16:40.785 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:16:40.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:16:40.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:16:40.849 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:16:40.849 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:16:40.849 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:16:40.849 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:16:40.849 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:16:40.849 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:40.849 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:40.849 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:16:40.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:16:40.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.1.asok injectdataerr pool-jerasure obj75 1 2026-02-19T13:16:40.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:665: TEST_ec_recovery_unfound: activate_osd td/test-erasure-eio 2 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:16:40.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:16:40.918 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:16:40.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:16:40.920 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:16:40.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:16:40.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:16:40.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:16:40.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:16:40.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:16:40.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:16:40.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:16:40.931 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:40.931+0000 7fc9df961900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:40.932 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:40.932+0000 7fc9df961900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:40.933 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:40.933+0000 7fc9df961900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:16:41.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:16:41.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:16:41.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:16:41.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:16:41.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:16:41.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:41.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:16:41.138 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:16:41.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:41.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:16:41.151 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:41.151+0000 7fc9df961900 -1 Falling back to public interface 2026-02-19T13:16:41.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:16:41.544 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:16:41.544+0000 7fc9df961900 -1 osd.2 41 log_to_monitors true 2026-02-19T13:16:42.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:16:42.358 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:16:42.358 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:16:42.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:16:42.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:16:42.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:16:42.578 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up out weight 0 up_from 47 up_thru 19 down_at 42 last_clean_interval [19,41) [v2:127.0.0.1:6818/1252534732,v1:127.0.0.1:6819/1252534732] [v2:127.0.0.1:6820/1252534732,v1:127.0.0.1:6821/1252534732] exists,up 155d6236-32ae-45b1-8ca8-5cb9a8af7b2b 2026-02-19T13:16:42.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:16:42.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:16:42.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:16:42.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:666: TEST_ec_recovery_unfound: ceph osd in 2 2026-02-19T13:16:42.846 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already in. 2026-02-19T13:16:42.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:668: TEST_ec_recovery_unfound: sleep 15 2026-02-19T13:16:57.860 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: seq 1 100 2026-02-19T13:16:57.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:670: TEST_ec_recovery_unfound: for tmp in $(seq 1 100) 2026-02-19T13:16:57.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: get_state 2.0 2026-02-19T13:16:57.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:60: get_state: local pgid=2.0 2026-02-19T13:16:57.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:61: get_state: local sname=state 2026-02-19T13:16:57.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:62: get_state: ceph --format json pg dump pgs 2026-02-19T13:16:57.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:63: get_state: jq -r '.pg_stats | .[] | select(.pgid=="2.0") | .state' 2026-02-19T13:16:58.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:671: TEST_ec_recovery_unfound: state=active+recovery_unfound+degraded 2026-02-19T13:16:58.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: echo active+recovery_unfound+degraded 2026-02-19T13:16:58.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:672: TEST_ec_recovery_unfound: grep recovery_unfound 2026-02-19T13:16:58.082 INFO:tasks.workunit.client.0.trial111.stdout:active+recovery_unfound+degraded 2026-02-19T13:16:58.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:673: TEST_ec_recovery_unfound: '[' 0 = 0 ']' 2026-02-19T13:16:58.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:674: TEST_ec_recovery_unfound: break 2026-02-19T13:16:58.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:680: TEST_ec_recovery_unfound: ceph pg dump pgs 2026-02-19T13:16:58.293 INFO:tasks.workunit.client.0.trial111.stdout:PG_STAT OBJECTS MISSING_ON_PRIMARY DEGRADED MISPLACED UNFOUND BYTES OMAP_BYTES* OMAP_KEYS* LOG LOG_DUPS DISK_LOG STATE STATE_STAMP VERSION REPORTED UP UP_PRIMARY ACTING ACTING_PRIMARY LAST_SCRUB SCRUB_STAMP LAST_DEEP_SCRUB DEEP_SCRUB_STAMP SNAPTRIMQ_LEN LAST_SCRUB_DURATION SCRUB_SCHEDULING OBJECTS_SCRUBBED OBJECTS_TRIMMED 2026-02-19T13:16:58.294 INFO:tasks.workunit.client.0.trial111.stdout:2.0 101 1 3 0 1 413696 0 0 101 0 101 active+recovery_unfound+degraded 2026-02-19T13:16:42.894247+0000 45'101 52:360 [3,1,4,0,2] 3 [3,1,4,0,2] 3 0'0 2026-02-19T13:16:30.792889+0000 0'0 2026-02-19T13:16:30.792889+0000 0 0 no scrub is scheduled 0 0 2026-02-19T13:16:58.294 INFO:tasks.workunit.client.0.trial111.stdout:1.3 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:42.892808+0000 0'0 52:60 [4,1,2] 4 [4,1,2] 4 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T22:58:39.897309+0000 0 0 2026-02-19T13:16:58.294 INFO:tasks.workunit.client.0.trial111.stdout:1.2 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:30.118508+0000 0'0 52:117 [0,4,1] 0 [0,4,1] 0 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T14:12:22.424617+0000 0 0 2026-02-19T13:16:58.294 INFO:tasks.workunit.client.0.trial111.stdout:1.1 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:30.419946+0000 0'0 52:42 [4,3,0] 4 [4,3,0] 4 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T17:04:49.581876+0000 0 0 2026-02-19T13:16:58.294 INFO:tasks.workunit.client.0.trial111.stdout:1.0 0 0 0 0 0 0 0 0 0 0 0 active+clean 2026-02-19T13:16:42.892911+0000 0'0 52:100 [1,0,2] 1 [1,0,2] 1 0'0 2026-02-19T13:16:15.699344+0000 0'0 2026-02-19T13:16:15.699344+0000 0 0 periodic scrub scheduled @ 2026-02-20T20:49:27.699844+0000 0 0 2026-02-19T13:16:58.294 INFO:tasks.workunit.client.0.trial111.stdout: 2026-02-19T13:16:58.294 INFO:tasks.workunit.client.0.trial111.stdout:* NOTE: Omap statistics are gathered during deep scrub and may be inaccurate soon afterwards depending on utilization. See http://docs.ceph.com/en/latest/dev/placement-group/#omap-statistics for further details. 2026-02-19T13:16:58.295 INFO:tasks.workunit.client.0.trial111.stderr:dumped pgs 2026-02-19T13:16:58.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:681: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:16:58.377 INFO:tasks.workunit.client.0.trial111.stdout:{ 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: "num_missing": 1, 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: "num_unfound": 1, 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: "objects": [ 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: "oid": { 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: "oid": "obj75", 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: "key": "", 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: "snapid": -2, 2026-02-19T13:16:58.378 INFO:tasks.workunit.client.0.trial111.stdout: "hash": 3565005477, 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "max": 0, 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "pool": 2, 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "namespace": "" 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "need": "45'76", 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "have": "0'0", 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "flags": "none", 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "clean_regions": "clean_offsets: [], clean_omap: false, new_object: true", 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "locations": [ 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:16:58.379 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:16:58.380 INFO:tasks.workunit.client.0.trial111.stdout: ] 2026-02-19T13:16:58.380 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.380 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.380 INFO:tasks.workunit.client.0.trial111.stdout: "state": "NotRecovering", 2026-02-19T13:16:58.380 INFO:tasks.workunit.client.0.trial111.stdout: "available_might_have_unfound": true, 2026-02-19T13:16:58.380 INFO:tasks.workunit.client.0.trial111.stdout: "might_have_unfound": [], 2026-02-19T13:16:58.381 INFO:tasks.workunit.client.0.trial111.stdout: "more": false 2026-02-19T13:16:58.381 INFO:tasks.workunit.client.0.trial111.stdout:} 2026-02-19T13:16:58.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:682: TEST_ec_recovery_unfound: ceph pg 2.0 query 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout:{ 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: "snap_trimq": "[]", 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: "snap_trimq_len": 0, 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+recovery_unfound+degraded", 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: "epoch": 52, 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.459 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: "acting_recovery_backfill": [ 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:16:58.460 INFO:tasks.workunit.client.0.trial111.stdout: "1(1)", 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "2(4)", 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "3(0)", 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "info": { 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s0", 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "0", 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'101", 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'75", 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "0'0", 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 101, 2026-02-19T13:16:58.461 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "51", 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 51, 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:16:58.462 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 51, 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 51, 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'101", 2026-02-19T13:16:58.463 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 361, 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 52, 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+recovery_unfound+degraded", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:16:42.894247+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:16:42.894247+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:16:42.894247+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:16:42.894247+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:16:37.181259+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:16:42.892503+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:16:42.892503+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:16:42.894247+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:16:42.890800+0000", 2026-02-19T13:16:58.464 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:16:42.894247+0000", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 51, 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "0'0", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "0'0", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.465 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 0, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:16:58.466 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 413696, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 101, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 505, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 1, 2026-02-19T13:16:58.467 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 1, 2026-02-19T13:16:58.468 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 3, 2026-02-19T13:16:58.468 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:16:58.468 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 1, 2026-02-19T13:16:58.468 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 101, 2026-02-19T13:16:58.468 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:16:58.468 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 101, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 404, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:16:58.469 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:16:58.470 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.471 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [ 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [ 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.472 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),1(1),2(4),3(0),4(2)", 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 100 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),4(2)", 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 1 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:16:58.473 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "peer_info": [ 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "peer": "0(3)", 2026-02-19T13:16:58.474 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s3", 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "3", 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'101", 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'101", 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "0'0", 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 101, 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:16:58.475 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:16:58.476 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:16:58.476 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:16:58.476 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.476 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 51, 2026-02-19T13:16:58.476 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.476 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:16:58.477 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:16:58.477 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:16:58.477 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 51, 2026-02-19T13:16:58.477 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 51, 2026-02-19T13:16:58.477 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:16:58.477 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.477 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.478 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.478 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.478 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.478 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:16:58.478 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.478 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'101", 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 349, 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 51, 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+undersized+degraded", 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:16:37.181259+0000", 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.479 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.480 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.480 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:16:37.592454+0000", 2026-02-19T13:16:58.480 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:16:37.592393+0000", 2026-02-19T13:16:58.480 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 51, 2026-02-19T13:16:58.480 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "0'0", 2026-02-19T13:16:58.480 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "0'0", 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:16:58.481 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 0, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:16:58.482 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T16:26:56.200123+0000", 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 413696, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 101, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 505, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 0, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 101, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:16:58.483 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 101, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 101, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 404, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:16:58.484 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:16:58.485 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.486 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [ 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: "3(0)", 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: "1(1)", 2026-02-19T13:16:58.487 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [ 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 101 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.488 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "peer": "1(1)", 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s1", 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "1", 2026-02-19T13:16:58.489 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'101", 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'101", 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "0'0", 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 101, 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 51, 2026-02-19T13:16:58.490 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 51, 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 51, 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.491 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'101", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 131, 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 45, 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+undersized+degraded", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:16:37.181259+0000", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.492 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:16:37.592454+0000", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:16:37.592393+0000", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 51, 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "0'0", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "0'0", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.493 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 0, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:16:58.494 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T16:26:56.200123+0000", 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 413696, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 101, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 505, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 1, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 101, 2026-02-19T13:16:58.495 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 101, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 101, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 404, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:16:58.496 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:16:58.497 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.498 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [ 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: "3(0)", 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: "1(1)", 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [ 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 101 2026-02-19T13:16:58.499 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.500 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "peer": "2(4)", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s4", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "4", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'101", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'75", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "0'0", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 101, 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T13:16:58.501 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 51, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 51, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 51, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.502 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'101", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 138, 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 49, 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "state": "remapped+peering", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:16:41.905174+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:16:41.805170+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:16:41.805170+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:16:41.702505+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:16:37.181259+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:16:41.905174+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:16:41.905174+0000", 2026-02-19T13:16:58.503 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:16:41.905174+0000", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 51, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "0'0", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "0'0", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 0, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:16:58.504 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "no scrub is scheduled", 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 413696, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 101, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 505, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:16:58.505 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 1, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 101, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 101, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 404, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:16:58.506 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.507 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [], 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [], 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:16:58.508 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "peer": "4(2)", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "pgid": "2.0s2", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "shard": "2", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "last_update": "45'101", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "last_complete": "45'101", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "log_tail": "0'0", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "last_user_version": 101, 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill": "MAX", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete": [], 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "partial_writes_last_complete_epoch": "0", 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [], 2026-02-19T13:16:58.509 INFO:tasks.workunit.client.0.trial111.stdout: "history": { 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_created": 39, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "epoch_pool_created": 39, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_started": 51, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_interval_clean": 39, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_split": 0, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_marked_full": 0, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "same_up_since": 51, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "same_interval_since": 51, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "same_primary_since": 39, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "prior_readable_until_ub": 0 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "stats": { 2026-02-19T13:16:58.510 INFO:tasks.workunit.client.0.trial111.stdout: "version": "45'101", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "reported_seq": 131, 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "reported_epoch": 45, 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "state": "active+undersized+degraded", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_fresh": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_change": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_active": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_peered": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean": "2026-02-19T13:16:37.181259+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_active": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_became_peered": "2026-02-19T13:16:37.593522+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_unstale": "2026-02-19T13:16:40.192211+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_undegraded": "2026-02-19T13:16:37.592454+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_fullsized": "2026-02-19T13:16:37.592393+0000", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "mapping_epoch": 51, 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "log_start": "0'0", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_start": "0'0", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "created": 39, 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_clean": 40, 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "parent": "0.0", 2026-02-19T13:16:58.511 INFO:tasks.workunit.client.0.trial111.stdout: "parent_split_bits": 0, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub": "0'0", 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub": "0'0", 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "last_deep_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "last_clean_scrub_stamp": "2026-02-19T13:16:30.792889+0000", 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "objects_scrubbed": 0, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "log_size": 101, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "log_dups_size": 0, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "ondisk_log_size": 101, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "stats_invalid": false, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "dirty_stats_invalid": false, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "omap_stats_invalid": false, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_stats_invalid": false, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "hitset_bytes_stats_invalid": false, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "pin_stats_invalid": false, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "manifest_stats_invalid": false, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrimq_len": 0, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "last_scrub_duration": 0, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_schedule": "periodic scrub scheduled @ 2026-02-20T16:26:56.200123+0000", 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_duration": 0, 2026-02-19T13:16:58.512 INFO:tasks.workunit.client.0.trial111.stdout: "objects_trimmed": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "snaptrim_duration": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "stat_sum": { 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes": 413696, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects": 101, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_clones": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_object_copies": 505, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing_on_primary": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_missing": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_degraded": 101, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_misplaced": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_unfound": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_dirty": 101, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_whiteouts": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_read": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_read_kb": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_write": 101, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_write_kb": 404, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_scrub_errors": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_shallow_scrub_errors": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_deep_scrub_errors": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_recovered": 0, 2026-02-19T13:16:58.513 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_recovered": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_keys_recovered": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_omap": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_hit_set_archive": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_bytes_hit_set_archive": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_kb": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_kb": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_promote": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_high": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_flush_mode_low": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_some": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_evict_mode_full": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_pinned": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_legacy_snapsets": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_large_omap_objects": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_manifest": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_bytes": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_omap_keys": 0, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "num_objects_repaired": 0 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: "up": [ 2026-02-19T13:16:58.514 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "acting": [ 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 3, 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 1, 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 4, 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 0, 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: 2 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "avail_no_missing": [ 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "3(0)", 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "0(3)", 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "1(1)", 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "4(2)" 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "object_location_counts": [ 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "shards": "0(3),1(1),3(0),4(2)", 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: "objects": 101 2026-02-19T13:16:58.515 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "blocked_by": [], 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "up_primary": 3, 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "acting_primary": 3, 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "purged_snaps": [] 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "empty": 0, 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "dne": 0, 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "incomplete": 0, 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "last_epoch_started": 52, 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "hit_set_history": { 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "current_last_update": "0'0", 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "history": [] 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "recovery_state": [ 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "name": "Started/Primary/Active", 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "enter_time": "2026-02-19T13:16:42.890673+0000", 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "might_have_unfound": [ 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.516 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "0(3)", 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "status": "already probed" 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "1(1)", 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "status": "already probed" 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "2(4)", 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "status": "already probed" 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "osd": "4(2)", 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "status": "already probed" 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "recovery_progress": { 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "backfill_targets": [], 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "waiting_on_backfill": [], 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "last_backfill_started": "MIN", 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "backfill_info": { 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "begin": "MIN", 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "end": "MIN", 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "objects": [] 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.517 INFO:tasks.workunit.client.0.trial111.stdout: "peer_backfill_info": [], 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "backfills_in_flight": [], 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "recovering": [], 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "pg_backend": { 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "recovery_ops": [], 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "read_ops": [] 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: { 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "name": "Started", 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "enter_time": "2026-02-19T13:16:42.791289+0000" 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: } 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: ], 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "scrubber": { 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "active": false, 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "must_scrub": false, 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "must_deep_scrub": false, 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "must_repair": false, 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "scrub_reg_stamp": "2026-02-20T16:26:56.200123+0000", 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "schedule": "not registered for scrubbing" 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: }, 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout: "agent_state": {} 2026-02-19T13:16:58.518 INFO:tasks.workunit.client.0.trial111.stdout:} 2026-02-19T13:16:58.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:16:58.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:684: TEST_ec_recovery_unfound: grep -q obj75 2026-02-19T13:16:58.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:16:58.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: jq .available_might_have_unfound 2026-02-19T13:16:58.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:686: TEST_ec_recovery_unfound: check=true 2026-02-19T13:16:58.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:687: TEST_ec_recovery_unfound: test true == true 2026-02-19T13:16:58.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: ceph pg 2.0 list_unfound 2026-02-19T13:16:58.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: jq '.might_have_unfound | length' 2026-02-19T13:16:58.707 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:689: TEST_ec_recovery_unfound: check=0 2026-02-19T13:16:58.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:690: TEST_ec_recovery_unfound: test 0 == 0 2026-02-19T13:16:58.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:693: TEST_ec_recovery_unfound: timeout 5 rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-19T13:17:03.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:694: TEST_ec_recovery_unfound: test 124 = 124 2026-02-19T13:17:03.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:696: TEST_ec_recovery_unfound: ceph pg 2.0 mark_unfound_lost delete 2026-02-19T13:17:03.785 INFO:tasks.workunit.client.0.trial111.stderr:pg has no unfound objects 2026-02-19T13:17:03.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:698: TEST_ec_recovery_unfound: wait_for_clean 2026-02-19T13:17:03.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:17:03.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:17:03.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:17:03.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:17:03.795 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:17:03.795 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:17:03.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:17:03.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:17:03.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:17:03.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:17:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:17:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:17:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:17:03.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:17:03.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:17:04.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:17:04.092 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:17:04.092 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:17:04.092 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:17:04.092 INFO:tasks.workunit.client.0.trial111.stderr:4' 2026-02-19T13:17:04.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:17:04.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:04.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:17:04.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803789 2026-02-19T13:17:04.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803789 2026-02-19T13:17:04.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789' 2026-02-19T13:17:04.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:04.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:17:04.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574861 2026-02-19T13:17:04.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574861 2026-02-19T13:17:04.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789 1-55834574861' 2026-02-19T13:17:04.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:04.253 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:17:04.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=201863462919 2026-02-19T13:17:04.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 201863462919 2026-02-19T13:17:04.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789 1-55834574861 2-201863462919' 2026-02-19T13:17:04.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:04.334 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:17:04.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149708 2026-02-19T13:17:04.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149708 2026-02-19T13:17:04.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789 1-55834574861 2-201863462919 3-111669149708' 2026-02-19T13:17:04.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:04.411 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:17:04.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=146028888075 2026-02-19T13:17:04.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 146028888075 2026-02-19T13:17:04.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803789 1-55834574861 2-201863462919 3-111669149708 4-146028888075' 2026-02-19T13:17:04.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:04.493 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803789 2026-02-19T13:17:04.493 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:04.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:17:04.495 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803789 2026-02-19T13:17:04.495 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:04.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803789 2026-02-19T13:17:04.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803789' 2026-02-19T13:17:04.496 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803789 2026-02-19T13:17:04.496 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:04.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-19T13:17:04.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:17:05.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:17:05.718 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803787 -lt 25769803789 2026-02-19T13:17:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:17:06.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:17:06.940 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:07.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803790 -lt 25769803789 2026-02-19T13:17:07.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:07.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574861 2026-02-19T13:17:07.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:07.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:17:07.165 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574861 2026-02-19T13:17:07.165 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:07.166 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574861 2026-02-19T13:17:07.166 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574861 2026-02-19T13:17:07.166 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574861' 2026-02-19T13:17:07.167 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:17:07.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574861 -lt 55834574861 2026-02-19T13:17:07.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:07.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-201863462919 2026-02-19T13:17:07.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:07.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:17:07.390 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-201863462919 2026-02-19T13:17:07.390 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:07.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=201863462919 2026-02-19T13:17:07.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 201863462919' 2026-02-19T13:17:07.391 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 201863462919 2026-02-19T13:17:07.392 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:17:07.612 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 201863462919 -lt 201863462919 2026-02-19T13:17:07.612 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:07.612 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149708 2026-02-19T13:17:07.613 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:07.613 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:17:07.614 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149708 2026-02-19T13:17:07.614 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:07.615 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149708 2026-02-19T13:17:07.615 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149708' 2026-02-19T13:17:07.615 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149708 2026-02-19T13:17:07.615 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:17:07.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149708 -lt 111669149708 2026-02-19T13:17:07.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:07.836 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-146028888075 2026-02-19T13:17:07.836 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:07.837 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:17:07.837 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-146028888075 2026-02-19T13:17:07.838 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:07.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=146028888075 2026-02-19T13:17:07.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 146028888075' 2026-02-19T13:17:07.839 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 146028888075 2026-02-19T13:17:07.839 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:17:08.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 146028888076 -lt 146028888075 2026-02-19T13:17:08.063 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:17:08.063 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:17:08.063 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:17:08.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:17:08.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:17:08.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:17:08.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:17:08.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:17:08.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:17:08.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:17:08.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:17:08.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:17:08.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:17:08.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:17:08.592 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:17:08.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:17:08.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:17:08.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:17:08.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: seq 1 100 2026-02-19T13:17:08.897 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:08.897 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj1 = obj75 ']' 2026-02-19T13:17:08.897 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj1 td/test-erasure-eio/CHECK 2026-02-19T13:17:08.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:08.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:08.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj2 = obj75 ']' 2026-02-19T13:17:08.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj2 td/test-erasure-eio/CHECK 2026-02-19T13:17:08.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:08.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:08.935 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj3 = obj75 ']' 2026-02-19T13:17:08.935 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj3 td/test-erasure-eio/CHECK 2026-02-19T13:17:08.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:08.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:08.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj4 = obj75 ']' 2026-02-19T13:17:08.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj4 td/test-erasure-eio/CHECK 2026-02-19T13:17:08.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:08.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:08.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj5 = obj75 ']' 2026-02-19T13:17:08.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj5 td/test-erasure-eio/CHECK 2026-02-19T13:17:08.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:08.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:08.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj6 = obj75 ']' 2026-02-19T13:17:08.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj6 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.008 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.008 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj7 = obj75 ']' 2026-02-19T13:17:09.008 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj7 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.025 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.026 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.026 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj8 = obj75 ']' 2026-02-19T13:17:09.026 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj8 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.043 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj9 = obj75 ']' 2026-02-19T13:17:09.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj9 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.060 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj10 = obj75 ']' 2026-02-19T13:17:09.062 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj10 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.078 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj11 = obj75 ']' 2026-02-19T13:17:09.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj11 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj12 = obj75 ']' 2026-02-19T13:17:09.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj12 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.114 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj13 = obj75 ']' 2026-02-19T13:17:09.115 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj13 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj14 = obj75 ']' 2026-02-19T13:17:09.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj14 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj15 = obj75 ']' 2026-02-19T13:17:09.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj15 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj16 = obj75 ']' 2026-02-19T13:17:09.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj16 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj17 = obj75 ']' 2026-02-19T13:17:09.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj17 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj18 = obj75 ']' 2026-02-19T13:17:09.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj18 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.223 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj19 = obj75 ']' 2026-02-19T13:17:09.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj19 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj20 = obj75 ']' 2026-02-19T13:17:09.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj20 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.259 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj21 = obj75 ']' 2026-02-19T13:17:09.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj21 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj22 = obj75 ']' 2026-02-19T13:17:09.279 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj22 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj23 = obj75 ']' 2026-02-19T13:17:09.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj23 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj24 = obj75 ']' 2026-02-19T13:17:09.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj24 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.332 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj25 = obj75 ']' 2026-02-19T13:17:09.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj25 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj26 = obj75 ']' 2026-02-19T13:17:09.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj26 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.368 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj27 = obj75 ']' 2026-02-19T13:17:09.370 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj27 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj28 = obj75 ']' 2026-02-19T13:17:09.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj28 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj29 = obj75 ']' 2026-02-19T13:17:09.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj29 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj30 = obj75 ']' 2026-02-19T13:17:09.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj30 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj31 = obj75 ']' 2026-02-19T13:17:09.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj31 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj32 = obj75 ']' 2026-02-19T13:17:09.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj32 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj33 = obj75 ']' 2026-02-19T13:17:09.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj33 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj34 = obj75 ']' 2026-02-19T13:17:09.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj34 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj35 = obj75 ']' 2026-02-19T13:17:09.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj35 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj36 = obj75 ']' 2026-02-19T13:17:09.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj36 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.548 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj37 = obj75 ']' 2026-02-19T13:17:09.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj37 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj38 = obj75 ']' 2026-02-19T13:17:09.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj38 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.585 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj39 = obj75 ']' 2026-02-19T13:17:09.586 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj39 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj40 = obj75 ']' 2026-02-19T13:17:09.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj40 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj41 = obj75 ']' 2026-02-19T13:17:09.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj41 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj42 = obj75 ']' 2026-02-19T13:17:09.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj42 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.658 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj43 = obj75 ']' 2026-02-19T13:17:09.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj43 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.676 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj44 = obj75 ']' 2026-02-19T13:17:09.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj44 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj45 = obj75 ']' 2026-02-19T13:17:09.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj45 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj46 = obj75 ']' 2026-02-19T13:17:09.714 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj46 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj47 = obj75 ']' 2026-02-19T13:17:09.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj47 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj48 = obj75 ']' 2026-02-19T13:17:09.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj48 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj49 = obj75 ']' 2026-02-19T13:17:09.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj49 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj50 = obj75 ']' 2026-02-19T13:17:09.785 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj50 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.802 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.803 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.803 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj51 = obj75 ']' 2026-02-19T13:17:09.803 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj51 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.821 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj52 = obj75 ']' 2026-02-19T13:17:09.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj52 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj53 = obj75 ']' 2026-02-19T13:17:09.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj53 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj54 = obj75 ']' 2026-02-19T13:17:09.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj54 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.875 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj55 = obj75 ']' 2026-02-19T13:17:09.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj55 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.892 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.894 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.894 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj56 = obj75 ']' 2026-02-19T13:17:09.894 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj56 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj57 = obj75 ']' 2026-02-19T13:17:09.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj57 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj58 = obj75 ']' 2026-02-19T13:17:09.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj58 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.946 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj59 = obj75 ']' 2026-02-19T13:17:09.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj59 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj60 = obj75 ']' 2026-02-19T13:17:09.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj60 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:09.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:09.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj61 = obj75 ']' 2026-02-19T13:17:09.984 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj61 td/test-erasure-eio/CHECK 2026-02-19T13:17:09.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj62 = obj75 ']' 2026-02-19T13:17:10.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj62 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj63 = obj75 ']' 2026-02-19T13:17:10.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj63 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj64 = obj75 ']' 2026-02-19T13:17:10.035 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj64 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj65 = obj75 ']' 2026-02-19T13:17:10.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj65 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj66 = obj75 ']' 2026-02-19T13:17:10.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj66 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj67 = obj75 ']' 2026-02-19T13:17:10.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj67 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj68 = obj75 ']' 2026-02-19T13:17:10.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj68 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj69 = obj75 ']' 2026-02-19T13:17:10.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj69 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj70 = obj75 ']' 2026-02-19T13:17:10.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj70 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.159 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj71 = obj75 ']' 2026-02-19T13:17:10.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj71 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj72 = obj75 ']' 2026-02-19T13:17:10.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj72 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj73 = obj75 ']' 2026-02-19T13:17:10.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj73 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj74 = obj75 ']' 2026-02-19T13:17:10.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj74 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj75 = obj75 ']' 2026-02-19T13:17:10.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:704: TEST_ec_recovery_unfound: rados -p pool-jerasure get obj75 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.249 INFO:tasks.workunit.client.0.trial111.stderr:error getting pool-jerasure/obj75: (2) No such file or directory 2026-02-19T13:17:10.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj76 = obj75 ']' 2026-02-19T13:17:10.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj76 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj77 = obj75 ']' 2026-02-19T13:17:10.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj77 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj78 = obj75 ']' 2026-02-19T13:17:10.286 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj78 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj79 = obj75 ']' 2026-02-19T13:17:10.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj79 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj80 = obj75 ']' 2026-02-19T13:17:10.322 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj80 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj81 = obj75 ']' 2026-02-19T13:17:10.342 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj81 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj82 = obj75 ']' 2026-02-19T13:17:10.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj82 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj83 = obj75 ']' 2026-02-19T13:17:10.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj83 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj84 = obj75 ']' 2026-02-19T13:17:10.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj84 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj85 = obj75 ']' 2026-02-19T13:17:10.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj85 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj86 = obj75 ']' 2026-02-19T13:17:10.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj86 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj87 = obj75 ']' 2026-02-19T13:17:10.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj87 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj88 = obj75 ']' 2026-02-19T13:17:10.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj88 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj89 = obj75 ']' 2026-02-19T13:17:10.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj89 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj90 = obj75 ']' 2026-02-19T13:17:10.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj90 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.518 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj91 = obj75 ']' 2026-02-19T13:17:10.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj91 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj92 = obj75 ']' 2026-02-19T13:17:10.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj92 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj93 = obj75 ']' 2026-02-19T13:17:10.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj93 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj94 = obj75 ']' 2026-02-19T13:17:10.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj94 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj95 = obj75 ']' 2026-02-19T13:17:10.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj95 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj96 = obj75 ']' 2026-02-19T13:17:10.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj96 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj97 = obj75 ']' 2026-02-19T13:17:10.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj97 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj98 = obj75 ']' 2026-02-19T13:17:10.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj98 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.661 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj99 = obj75 ']' 2026-02-19T13:17:10.662 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj99 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.679 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:700: TEST_ec_recovery_unfound: for i in $(seq 1 $lastobj) 2026-02-19T13:17:10.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:702: TEST_ec_recovery_unfound: '[' obj100 = obj75 ']' 2026-02-19T13:17:10.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:706: TEST_ec_recovery_unfound: rados --pool pool-jerasure get obj100 td/test-erasure-eio/CHECK 2026-02-19T13:17:10.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:707: TEST_ec_recovery_unfound: diff -q td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.698 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:711: TEST_ec_recovery_unfound: rm -f td/test-erasure-eio/ORIGINAL td/test-erasure-eio/CHECK 2026-02-19T13:17:10.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:713: TEST_ec_recovery_unfound: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:17:10.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:17:10.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:17:10.959 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:17:10.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:17:11.239 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:17:11.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:17:11.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:17:11.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:17:11.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:17:11.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:11.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:11.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:11.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:11.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:11.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:11.360 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:17:11.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:17:11.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:17:11.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:17:11.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:17:11.363 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:17:11.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:11.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:17:11.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:17:11.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:11.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:17:11.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:17:11.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:17:11.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:17:11.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:11.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:11.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:17:11.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:17:11.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:17:11.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:11.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:11.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:11.390 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:17:11.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:17:11.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:17:11.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:17:11.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:17:11.393 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:17:11.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:11.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:17:11.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:17:11.394 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:11.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:17:11.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:17:11.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:17:11.397 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:17:11.397 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:11.397 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:11.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:17:11.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:17:11.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:17:11.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:17:11.399 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:17:11.399 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:11.399 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:11.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:17:11.400 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:17:11.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:17:11.421 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:17:11.421 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:11.421 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:11.421 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:11.421 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:11.421 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:11.421 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:11.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:17:11.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:17:11.446 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:17:11.446 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:17:11.446 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:17:11.446 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:17:11.447 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:17:11.447 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:17:11.447 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:17:11.447 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:17:11.448 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:11.448 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:11.448 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:11.448 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:17:11.448 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:17:11.448 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:17:11.507 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:17:11.507 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:17:11.507 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:17:11.507 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:17:11.507 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:17:11.508 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:17:11.509 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:17:11.509 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:17:11.509 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:11.509 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:11.509 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:11.509 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:17:11.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:17:11.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:17:11.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:17:11.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:17:11.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:17:11.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:17:11.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:17:11.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:17:11.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:17:11.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:17:11.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:11.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:11.680 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:11.680 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:11.680 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:11.680 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:11.680 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:17:11.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:17:11.698 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:17:11.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:17:11.801 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:17:11.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:17:12.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:17:12.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:17:12.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:17:12.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:12.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:12.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:12.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:17:12.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:17:12.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:17:12.870 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:17:12.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:17:12.879 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:17:11.438+0000 7f88ee059dc0 0 load: jerasure load: lrc 2026-02-19T13:17:12.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_ec_single_recovery_error td/test-erasure-eio 2026-02-19T13:17:12.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:410: TEST_ec_single_recovery_error: local dir=td/test-erasure-eio 2026-02-19T13:17:12.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:411: TEST_ec_single_recovery_error: local objname=myobject 2026-02-19T13:17:12.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:413: TEST_ec_single_recovery_error: setup_osds 7 2026-02-19T13:17:12.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=7 2026-02-19T13:17:12.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:17:12.880 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 7 - 1 2026-02-19T13:17:12.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 6 2026-02-19T13:17:12.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:12.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:17:12.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:12.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:12.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:17:12.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:12.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:12.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:12.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:17:12.885 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:12.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=84943fef-9eda-40ca-bc99-96f842ed2f1e 2026-02-19T13:17:12.885 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 84943fef-9eda-40ca-bc99-96f842ed2f1e 2026-02-19T13:17:12.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 84943fef-9eda-40ca-bc99-96f842ed2f1e' 2026-02-19T13:17:12.885 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:12.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBYDZdpHC1yNRAAjdWCcsJ4o8DfqoaO7nBlXA== 2026-02-19T13:17:12.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBYDZdpHC1yNRAAjdWCcsJ4o8DfqoaO7nBlXA=="}' 2026-02-19T13:17:12.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 84943fef-9eda-40ca-bc99-96f842ed2f1e -i td/test-erasure-eio/0/new.json 2026-02-19T13:17:13.117 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:13.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:17:13.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBYDZdpHC1yNRAAjdWCcsJ4o8DfqoaO7nBlXA== --osd-uuid 84943fef-9eda-40ca-bc99-96f842ed2f1e 2026-02-19T13:17:13.145 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:13.145+0000 7f2ec338f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:13.147 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:13.147+0000 7f2ec338f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:13.147 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:13.148+0000 7f2ec338f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:13.148 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:13.148+0000 7f2ec338f900 -1 bdev(0x558774506c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:13.148 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:13.148+0000 7f2ec338f900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:17:13.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:17:13.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:13.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:17:13.567 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:17:13.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:13.866 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:17:13.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:17:13.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:13.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:13.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:13.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:13.880 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:13.881+0000 7f9e7b368900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:13.882 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:13.882+0000 7f9e7b368900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:13.882 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:13.883+0000 7f9e7b368900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:14.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:17:14.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:14.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:17:14.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:14.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:14.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:14.087 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:14.088 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:14.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:14.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:17:14.109 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:14.110+0000 7f9e7b368900 -1 Falling back to public interface 2026-02-19T13:17:14.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:14.417 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:14.417+0000 7f9e7b368900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:17:15.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:15.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:15.309 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:15.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:15.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:15.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:17:15.529 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3108260108,v1:127.0.0.1:6803/3108260108] [v2:127.0.0.1:6804/3108260108,v1:127.0.0.1:6805/3108260108] exists,up 84943fef-9eda-40ca-bc99-96f842ed2f1e 2026-02-19T13:17:15.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:15.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:15.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:15.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:15.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:17:15.536 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:15.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=e094100a-4990-47ae-9a9b-1a595ded7b06 2026-02-19T13:17:15.537 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 e094100a-4990-47ae-9a9b-1a595ded7b06 2026-02-19T13:17:15.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 e094100a-4990-47ae-9a9b-1a595ded7b06' 2026-02-19T13:17:15.538 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:15.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBbDZdpsIeOIBAAa4mJeWMwrqElNdS6N+HevA== 2026-02-19T13:17:15.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBbDZdpsIeOIBAAa4mJeWMwrqElNdS6N+HevA=="}' 2026-02-19T13:17:15.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new e094100a-4990-47ae-9a9b-1a595ded7b06 -i td/test-erasure-eio/1/new.json 2026-02-19T13:17:15.767 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:15.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:17:15.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBbDZdpsIeOIBAAa4mJeWMwrqElNdS6N+HevA== --osd-uuid e094100a-4990-47ae-9a9b-1a595ded7b06 2026-02-19T13:17:15.796 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:15.796+0000 7fc3c6568900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:15.797 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:15.797+0000 7fc3c6568900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:15.798 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:15.798+0000 7fc3c6568900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:15.798 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:15.799+0000 7fc3c6568900 -1 bdev(0x56361c32ac00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:15.798 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:15.799+0000 7fc3c6568900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:17:16.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:17:16.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:16.223 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:17:16.223 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:17:16.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:16.527 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:17:16.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:17:16.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:16.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:16.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:16.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:16.543 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:16.543+0000 7fceb448d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:16.544 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:16.544+0000 7fceb448d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:16.545 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:16.545+0000 7fceb448d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:16.740 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:16.740+0000 7fceb448d900 -1 Falling back to public interface 2026-02-19T13:17:16.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:17:16.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:16.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:17:16.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:16.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:16.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:16.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:16.753 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:16.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:16.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:17:16.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:17.072 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:17.072+0000 7fceb448d900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:17:17.973 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:17.973 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:17.974 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:17.974 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:17.974 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:17.974 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:17:18.196 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2821069436,v1:127.0.0.1:6811/2821069436] [v2:127.0.0.1:6812/2821069436,v1:127.0.0.1:6813/2821069436] exists,up e094100a-4990-47ae-9a9b-1a595ded7b06 2026-02-19T13:17:18.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:17:18.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:18.202 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:18.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:18.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:18.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:18.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:18.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:17:18.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:18.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3af11e12-216d-4a24-91b8-db4200682960 2026-02-19T13:17:18.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 3af11e12-216d-4a24-91b8-db4200682960' 2026-02-19T13:17:18.204 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 3af11e12-216d-4a24-91b8-db4200682960 2026-02-19T13:17:18.205 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:18.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBeDZdp7QCtDBAAg1aMmmNwbHxGTF0bErr8Ug== 2026-02-19T13:17:18.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBeDZdp7QCtDBAAg1aMmmNwbHxGTF0bErr8Ug=="}' 2026-02-19T13:17:18.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3af11e12-216d-4a24-91b8-db4200682960 -i td/test-erasure-eio/2/new.json 2026-02-19T13:17:18.432 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:17:18.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:17:18.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBeDZdp7QCtDBAAg1aMmmNwbHxGTF0bErr8Ug== --osd-uuid 3af11e12-216d-4a24-91b8-db4200682960 2026-02-19T13:17:18.461 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:18.461+0000 7f32ef0b2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:18.462 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:18.462+0000 7f32ef0b2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:18.463 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:18.463+0000 7f32ef0b2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:18.463 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:18.464+0000 7f32ef0b2900 -1 bdev(0x55e0b7984c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:18.464 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:18.464+0000 7f32ef0b2900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:17:18.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:17:18.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:18.912 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:17:18.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:17:18.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:19.212 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:17:19.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:17:19.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:19.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:19.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:19.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:19.228 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:19.228+0000 7fefe8a30900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:19.229 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:19.230+0000 7fefe8a30900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:19.230 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:19.231+0000 7fefe8a30900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:19.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:19.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:17:19.454 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:19.455+0000 7fefe8a30900 -1 Falling back to public interface 2026-02-19T13:17:19.652 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:19.810 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:19.810+0000 7fefe8a30900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:17:20.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:20.654 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:20.654 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:20.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:20.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:20.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:17:20.875 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4113526369,v1:127.0.0.1:6819/4113526369] [v2:127.0.0.1:6820/4113526369,v1:127.0.0.1:6821/4113526369] exists,up 3af11e12-216d-4a24-91b8-db4200682960 2026-02-19T13:17:20.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:20.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:20.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:20.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:20.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:20.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:17:20.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:20.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=3dc48bac-d2a0-4d37-af4e-d64933ab265d 2026-02-19T13:17:20.883 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 3dc48bac-d2a0-4d37-af4e-d64933ab265d 2026-02-19T13:17:20.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 3dc48bac-d2a0-4d37-af4e-d64933ab265d' 2026-02-19T13:17:20.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:20.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBgDZdpW6MdNRAA3YzwRx9eMZzjHltRwjKyzQ== 2026-02-19T13:17:20.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBgDZdpW6MdNRAA3YzwRx9eMZzjHltRwjKyzQ=="}' 2026-02-19T13:17:20.890 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 3dc48bac-d2a0-4d37-af4e-d64933ab265d -i td/test-erasure-eio/3/new.json 2026-02-19T13:17:21.108 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:17:21.119 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:17:21.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBgDZdpW6MdNRAA3YzwRx9eMZzjHltRwjKyzQ== --osd-uuid 3dc48bac-d2a0-4d37-af4e-d64933ab265d 2026-02-19T13:17:21.135 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:21.135+0000 7fd478978900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:21.136 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:21.137+0000 7fd478978900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:21.137 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:21.137+0000 7fd478978900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:21.138 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:21.138+0000 7fd478978900 -1 bdev(0x564960d88c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:21.138 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:21.138+0000 7fd478978900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:17:21.534 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:17:21.535 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:21.535 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:17:21.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:17:21.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:21.835 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:17:21.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:17:21.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:21.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:21.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:21.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:21.849 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:21.849+0000 7f9c872e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:21.850 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:21.851+0000 7f9c872e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:21.851 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:21.852+0000 7f9c872e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:22.054 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:22.054+0000 7f9c872e4900 -1 Falling back to public interface 2026-02-19T13:17:22.057 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:17:22.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:22.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:17:22.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:22.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:22.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:22.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:22.058 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:22.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:22.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:17:22.276 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:22.401 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:22.401+0000 7f9c872e4900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:17:23.277 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:23.277 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:23.278 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:23.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:23.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:23.278 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:17:23.504 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/4282861283,v1:127.0.0.1:6827/4282861283] [v2:127.0.0.1:6828/4282861283,v1:127.0.0.1:6829/4282861283] exists,up 3dc48bac-d2a0-4d37-af4e-d64933ab265d 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 4 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=4 2026-02-19T13:17:23.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/4 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/4' 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/4/journal' 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:23.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:23.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/4 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ab030f37-5af6-4f48-883c-f80bd7940532 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd4 ab030f37-5af6-4f48-883c-f80bd7940532' 2026-02-19T13:17:23.512 INFO:tasks.workunit.client.0.trial111.stdout:add osd4 ab030f37-5af6-4f48-883c-f80bd7940532 2026-02-19T13:17:23.513 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:23.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBjDZdp1y4JHxAAYRwsBpNp+156/8K/sQBsoQ== 2026-02-19T13:17:23.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBjDZdp1y4JHxAAYRwsBpNp+156/8K/sQBsoQ=="}' 2026-02-19T13:17:23.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ab030f37-5af6-4f48-883c-f80bd7940532 -i td/test-erasure-eio/4/new.json 2026-02-19T13:17:23.747 INFO:tasks.workunit.client.0.trial111.stdout:4 2026-02-19T13:17:23.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/4/new.json 2026-02-19T13:17:23.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBjDZdp1y4JHxAAYRwsBpNp+156/8K/sQBsoQ== --osd-uuid ab030f37-5af6-4f48-883c-f80bd7940532 2026-02-19T13:17:23.774 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:23.775+0000 7f679830e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:23.776 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:23.776+0000 7f679830e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:23.776 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:23.777+0000 7f679830e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:23.777 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:23.777+0000 7f679830e900 -1 bdev(0x55d5dc986c00 td/test-erasure-eio/4/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:23.777 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:23.777+0000 7f679830e900 -1 bluestore(td/test-erasure-eio/4) _read_fsid unparsable uuid 2026-02-19T13:17:24.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/4/keyring 2026-02-19T13:17:24.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:24.164 INFO:tasks.workunit.client.0.trial111.stdout:adding osd4 key to auth repository 2026-02-19T13:17:24.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd4 key to auth repository 2026-02-19T13:17:24.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/4/keyring auth add osd.4 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:24.465 INFO:tasks.workunit.client.0.trial111.stdout:start osd.4 2026-02-19T13:17:24.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.4 2026-02-19T13:17:24.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 4 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/4 --osd-journal=td/test-erasure-eio/4/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:24.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:24.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:24.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:24.481 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:24.481+0000 7f2398e39900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:24.482 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:24.482+0000 7f2398e39900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:24.483 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:24.483+0000 7f2398e39900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:24.667 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:24.668+0000 7f2398e39900 -1 Falling back to public interface 2026-02-19T13:17:24.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 4 2026-02-19T13:17:24.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:24.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=4 2026-02-19T13:17:24.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:24.688 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:24.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:24.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:24.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:24.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:24.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:17:24.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:25.077 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:25.078+0000 7f2398e39900 -1 osd.4 0 log_to_monitors true 2026-02-19T13:17:25.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:25.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:25.913 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:25.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:25.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:25.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.4 up' 2026-02-19T13:17:26.130 INFO:tasks.workunit.client.0.trial111.stdout:osd.4 up in weight 1 up_from 35 up_thru 36 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6834/711310236,v1:127.0.0.1:6835/711310236] [v2:127.0.0.1:6836/711310236,v1:127.0.0.1:6837/711310236] exists,up ab030f37-5af6-4f48-883c-f80bd7940532 2026-02-19T13:17:26.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:26.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:26.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:26.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:26.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 5 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=5 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/5 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/5' 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/5/journal' 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:26.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:26.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/5 2026-02-19T13:17:26.133 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:26.133 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=606dc4b3-5377-496e-9626-d3569cc04f0f 2026-02-19T13:17:26.133 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd5 606dc4b3-5377-496e-9626-d3569cc04f0f' 2026-02-19T13:17:26.134 INFO:tasks.workunit.client.0.trial111.stdout:add osd5 606dc4b3-5377-496e-9626-d3569cc04f0f 2026-02-19T13:17:26.134 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:26.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBmDZdpzPy3CBAApR5eNYBQPKkaPdAvT9tsnQ== 2026-02-19T13:17:26.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBmDZdpzPy3CBAApR5eNYBQPKkaPdAvT9tsnQ=="}' 2026-02-19T13:17:26.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 606dc4b3-5377-496e-9626-d3569cc04f0f -i td/test-erasure-eio/5/new.json 2026-02-19T13:17:26.365 INFO:tasks.workunit.client.0.trial111.stdout:5 2026-02-19T13:17:26.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/5/new.json 2026-02-19T13:17:26.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBmDZdpzPy3CBAApR5eNYBQPKkaPdAvT9tsnQ== --osd-uuid 606dc4b3-5377-496e-9626-d3569cc04f0f 2026-02-19T13:17:26.393 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:26.393+0000 7f0f8c7c5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:26.394 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:26.394+0000 7f0f8c7c5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:26.395 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:26.395+0000 7f0f8c7c5900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:26.396 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:26.395+0000 7f0f8c7c5900 -1 bdev(0x558e5d1cec00 td/test-erasure-eio/5/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:26.396 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:26.395+0000 7f0f8c7c5900 -1 bluestore(td/test-erasure-eio/5) _read_fsid unparsable uuid 2026-02-19T13:17:26.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/5/keyring 2026-02-19T13:17:26.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:26.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd5 key to auth repository 2026-02-19T13:17:26.811 INFO:tasks.workunit.client.0.trial111.stdout:adding osd5 key to auth repository 2026-02-19T13:17:26.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/5/keyring auth add osd.5 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:27.118 INFO:tasks.workunit.client.0.trial111.stdout:start osd.5 2026-02-19T13:17:27.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.5 2026-02-19T13:17:27.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 5 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/5 --osd-journal=td/test-erasure-eio/5/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:27.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:27.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:27.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:27.133 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:27.133+0000 7f6a4dea3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:27.134 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:27.134+0000 7f6a4dea3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:27.135 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:27.135+0000 7f6a4dea3900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:27.327 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:27.327+0000 7f6a4dea3900 -1 Falling back to public interface 2026-02-19T13:17:27.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 5 2026-02-19T13:17:27.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:27.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=5 2026-02-19T13:17:27.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:27.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:27.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:27.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:27.337 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:27.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:27.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:17:27.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:27.751 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:27.751+0000 7f6a4dea3900 -1 osd.5 0 log_to_monitors true 2026-02-19T13:17:28.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:28.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:28.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:28.557 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:28.557 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:28.557 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.5 up' 2026-02-19T13:17:28.777 INFO:tasks.workunit.client.0.trial111.stdout:osd.5 up in weight 1 up_from 43 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6842/3065008220,v1:127.0.0.1:6843/3065008220] [v2:127.0.0.1:6844/3065008220,v1:127.0.0.1:6845/3065008220] exists,up 606dc4b3-5377-496e-9626-d3569cc04f0f 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 6 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:28.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=6 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/6 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/6' 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/6/journal' 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:28.782 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:28.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/6 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=54f774a1-29c7-4442-b588-4ce84177d248 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd6 54f774a1-29c7-4442-b588-4ce84177d248' 2026-02-19T13:17:28.784 INFO:tasks.workunit.client.0.trial111.stdout:add osd6 54f774a1-29c7-4442-b588-4ce84177d248 2026-02-19T13:17:28.785 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:28.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBoDZdpr1xJLxAAgQ8iXTjp9SO9aG+SEp0tkQ== 2026-02-19T13:17:28.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBoDZdpr1xJLxAAgQ8iXTjp9SO9aG+SEp0tkQ=="}' 2026-02-19T13:17:28.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 54f774a1-29c7-4442-b588-4ce84177d248 -i td/test-erasure-eio/6/new.json 2026-02-19T13:17:29.013 INFO:tasks.workunit.client.0.trial111.stdout:6 2026-02-19T13:17:29.025 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/6/new.json 2026-02-19T13:17:29.026 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBoDZdpr1xJLxAAgQ8iXTjp9SO9aG+SEp0tkQ== --osd-uuid 54f774a1-29c7-4442-b588-4ce84177d248 2026-02-19T13:17:29.042 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.042+0000 7fa8f2849900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:29.043 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.043+0000 7fa8f2849900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:29.044 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.044+0000 7fa8f2849900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:29.044 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.045+0000 7fa8f2849900 -1 bdev(0x55eca4770c00 td/test-erasure-eio/6/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:29.044 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.045+0000 7fa8f2849900 -1 bluestore(td/test-erasure-eio/6) _read_fsid unparsable uuid 2026-02-19T13:17:29.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/6/keyring 2026-02-19T13:17:29.484 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:29.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd6 key to auth repository 2026-02-19T13:17:29.486 INFO:tasks.workunit.client.0.trial111.stdout:adding osd6 key to auth repository 2026-02-19T13:17:29.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/6/keyring auth add osd.6 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:29.785 INFO:tasks.workunit.client.0.trial111.stdout:start osd.6 2026-02-19T13:17:29.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.6 2026-02-19T13:17:29.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 6 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/6 --osd-journal=td/test-erasure-eio/6/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:29.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:29.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:29.786 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:29.800 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.800+0000 7f9d6d064900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:29.801 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.802+0000 7f9d6d064900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:29.802 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.803+0000 7f9d6d064900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:29.993 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:29.993+0000 7f9d6d064900 -1 Falling back to public interface 2026-02-19T13:17:30.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 6 2026-02-19T13:17:30.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:30.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=6 2026-02-19T13:17:30.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:30.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:30.005 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:30.005 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:30.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:30.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:30.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:17:30.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:30.359 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:30.359+0000 7f9d6d064900 -1 osd.6 0 log_to_monitors true 2026-02-19T13:17:31.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:31.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:31.225 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:31.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:31.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:31.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.6 up' 2026-02-19T13:17:31.450 INFO:tasks.workunit.client.0.trial111.stdout:osd.6 up in weight 1 up_from 49 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6850/1440423973,v1:127.0.0.1:6851/1440423973] [v2:127.0.0.1:6852/1440423973,v1:127.0.0.1:6853/1440423973] exists,up 54f774a1-29c7-4442-b588-4ce84177d248 2026-02-19T13:17:31.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:17:31.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:17:31.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:17:31.509 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:17:31.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:17:31.518 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:17:14.113+0000 7f9e7b368900 0 load: jerasure load: lrc 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:415: TEST_ec_single_recovery_error: local poolname=pool-jerasure 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:416: TEST_ec_single_recovery_error: create_erasure_coded_pool pool-jerasure 3 2 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=3 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=2 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:17:31.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=3 m=2 crush-failure-domain=osd 2026-02-19T13:17:31.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:17:31.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:17:32.158 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:17:32.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:17:33.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:17:33.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:17:33.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:17:33.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:17:33.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:17:33.173 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:17:33.173 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:17:33.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:17:33.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:17:33.173 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:17:33.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:17:33.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:17:33.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:17:33.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:17:33.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:17:33.250 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:17:33.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:33.467 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:17:33.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-19T13:17:33.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-19T13:17:33.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782' 2026-02-19T13:17:33.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:33.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:17:33.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574854 2026-02-19T13:17:33.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574854 2026-02-19T13:17:33.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854' 2026-02-19T13:17:33.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:33.625 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:17:33.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378629 2026-02-19T13:17:33.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378629 2026-02-19T13:17:33.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629' 2026-02-19T13:17:33.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:33.706 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:17:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116997 2026-02-19T13:17:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116997 2026-02-19T13:17:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-115964116997' 2026-02-19T13:17:33.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:33.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:17:33.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855364 2026-02-19T13:17:33.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855364 2026-02-19T13:17:33.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-115964116997 4-150323855364' 2026-02-19T13:17:33.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:33.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:17:33.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593732 2026-02-19T13:17:33.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593732 2026-02-19T13:17:33.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-115964116997 4-150323855364 5-184683593732' 2026-02-19T13:17:33.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:33.941 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:17:34.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-19T13:17:34.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-19T13:17:34.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-55834574854 2-81604378629 3-115964116997 4-150323855364 5-184683593732 6-210453397507' 2026-02-19T13:17:34.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:34.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-19T13:17:34.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:34.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:17:34.022 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803782 2026-02-19T13:17:34.022 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:34.023 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803782 2026-02-19T13:17:34.024 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803782 2026-02-19T13:17:34.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-19T13:17:34.024 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-19T13:17:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:34.248 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574854 2026-02-19T13:17:34.248 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:34.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:17:34.249 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574854 2026-02-19T13:17:34.249 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:34.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574854 2026-02-19T13:17:34.250 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574854 2026-02-19T13:17:34.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574854' 2026-02-19T13:17:34.251 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:17:34.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574854 -lt 55834574854 2026-02-19T13:17:34.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:34.473 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378629 2026-02-19T13:17:34.473 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:34.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:17:34.474 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-81604378629 2026-02-19T13:17:34.474 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:34.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378629 2026-02-19T13:17:34.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 81604378629' 2026-02-19T13:17:34.475 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 81604378629 2026-02-19T13:17:34.476 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:17:34.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378629 -lt 81604378629 2026-02-19T13:17:34.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:34.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116997 2026-02-19T13:17:34.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:34.698 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:17:34.699 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964116997 2026-02-19T13:17:34.699 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:34.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116997 2026-02-19T13:17:34.700 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 115964116997 2026-02-19T13:17:34.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964116997' 2026-02-19T13:17:34.701 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:17:34.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116997 -lt 115964116997 2026-02-19T13:17:34.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:34.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855364 2026-02-19T13:17:34.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:34.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:17:34.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855364 2026-02-19T13:17:34.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:34.924 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855364 2026-02-19T13:17:34.924 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 150323855364 2026-02-19T13:17:34.925 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855364' 2026-02-19T13:17:34.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:17:35.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855364 -lt 150323855364 2026-02-19T13:17:35.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:35.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593732 2026-02-19T13:17:35.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:35.148 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:17:35.149 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-184683593732 2026-02-19T13:17:35.149 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:35.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593732 2026-02-19T13:17:35.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 184683593732' 2026-02-19T13:17:35.151 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 184683593732 2026-02-19T13:17:35.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:17:35.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593732 -lt 184683593732 2026-02-19T13:17:35.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:35.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 6-210453397507 2026-02-19T13:17:35.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:35.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=6 2026-02-19T13:17:35.377 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 6-210453397507 2026-02-19T13:17:35.377 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:35.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397507 2026-02-19T13:17:35.378 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.6 seq 210453397507 2026-02-19T13:17:35.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.6 seq 210453397507' 2026-02-19T13:17:35.378 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 6 2026-02-19T13:17:35.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-19T13:17:35.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:17:35.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:17:35.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:17:35.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:17:35.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:17:35.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:17:35.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:17:35.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:17:35.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:17:35.896 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:17:35.897 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:17:36.111 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:17:36.111 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:17:36.111 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:17:36.111 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:418: TEST_ec_single_recovery_error: rados_put td/test-erasure-eio pool-jerasure myobject 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=myobject 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:17:36.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put myobject td/test-erasure-eio/ORIGINAL 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:419: TEST_ec_single_recovery_error: inject_eio ec data pool-jerasure myobject td/test-erasure-eio 0 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=myobject 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure myobject 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:17:36.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:17:36.432 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:17:36.432 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:17:36.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:17:36.656 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:17:36.656 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:17:36.656 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:17:36.656 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:17:36.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-19T13:17:36.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '5' '1' '0' '6') 2026-02-19T13:17:36.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:17:36.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-19T13:17:36.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:17:36.657 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T13:17:36.659 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:17:36.660 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:17:36.660 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:36.660 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:36.660 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:36.660 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:17:36.660 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:17:36.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:17:36.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:17:36.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:17:36.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-19T13:17:36.727 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:17:36.727 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:17:36.727 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:36.727 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:36.727 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:36.727 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:17:36.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:17:36.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.3.asok injectdataerr pool-jerasure myobject 0 2026-02-19T13:17:36.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: get_osds pool-jerasure myobject 2026-02-19T13:17:36.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:17:36.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=myobject 2026-02-19T13:17:36.794 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure myobject 2026-02-19T13:17:36.794 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:17:37.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:17:37.017 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:17:37.017 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:17:37.017 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:17:37.017 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 5 1 0 6 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: initial_osds=('3' '5' '1' '0' '6') 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:421: TEST_ec_single_recovery_error: local -a initial_osds 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:422: TEST_ec_single_recovery_error: local last_osd=6 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:424: TEST_ec_single_recovery_error: kill_daemons td/test-erasure-eio TERM osd.6 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:37.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:37.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:37.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:37.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:425: TEST_ec_single_recovery_error: ceph osd down 6 2026-02-19T13:17:37.539 INFO:tasks.workunit.client.0.trial111.stderr:osd.6 is already down. 2026-02-19T13:17:37.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:426: TEST_ec_single_recovery_error: ceph osd out 6 2026-02-19T13:17:37.792 INFO:tasks.workunit.client.0.trial111.stderr:osd.6 is already out. 2026-02-19T13:17:37.804 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:429: TEST_ec_single_recovery_error: wait_for_clean 2026-02-19T13:17:37.804 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:17:37.804 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:17:37.804 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:17:37.804 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:17:37.804 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:17:37.805 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:17:37.805 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:17:37.805 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:17:37.805 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:17:37.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:17:37.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:17:37.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:17:37.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:17:37.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:17:37.886 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:17:38.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:17:38.108 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:17:38.108 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:17:38.108 INFO:tasks.workunit.client.0.trial111.stderr:3 2026-02-19T13:17:38.108 INFO:tasks.workunit.client.0.trial111.stderr:4 2026-02-19T13:17:38.108 INFO:tasks.workunit.client.0.trial111.stderr:5 2026-02-19T13:17:38.109 INFO:tasks.workunit.client.0.trial111.stderr:6' 2026-02-19T13:17:38.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:17:38.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:38.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:17:38.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-19T13:17:38.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-19T13:17:38.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785' 2026-02-19T13:17:38.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:38.188 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:17:38.266 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574857 2026-02-19T13:17:38.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574857 2026-02-19T13:17:38.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857' 2026-02-19T13:17:38.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:38.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:17:38.349 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378632 2026-02-19T13:17:38.349 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378632 2026-02-19T13:17:38.349 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632' 2026-02-19T13:17:38.349 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:38.349 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:17:38.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116999 2026-02-19T13:17:38.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116999 2026-02-19T13:17:38.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-115964116999' 2026-02-19T13:17:38.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:38.427 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.4 flush_pg_stats 2026-02-19T13:17:38.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=150323855367 2026-02-19T13:17:38.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 150323855367 2026-02-19T13:17:38.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-115964116999 4-150323855367' 2026-02-19T13:17:38.506 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:38.507 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.5 flush_pg_stats 2026-02-19T13:17:38.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593734 2026-02-19T13:17:38.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593734 2026-02-19T13:17:38.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-55834574857 2-81604378632 3-115964116999 4-150323855367 5-184683593734' 2026-02-19T13:17:38.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:38.588 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.6 flush_pg_stats 2026-02-19T13:17:38.645 INFO:tasks.workunit.client.0.trial111.stderr:Error ENXIO: problem getting command descriptions from osd.6 2026-02-19T13:17:38.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq= 2026-02-19T13:17:38.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z '' 2026-02-19T13:17:38.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2261: flush_pg_stats: continue 2026-02-19T13:17:38.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:38.648 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-19T13:17:38.648 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:38.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:17:38.649 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803785 2026-02-19T13:17:38.649 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:38.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803785 2026-02-19T13:17:38.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-19T13:17:38.651 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803785 2026-02-19T13:17:38.651 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:38.868 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-19T13:17:38.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:17:39.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:17:39.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:40.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803783 -lt 25769803785 2026-02-19T13:17:40.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:17:41.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:17:41.094 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:41.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-19T13:17:41.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:41.315 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574857 2026-02-19T13:17:41.315 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:41.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:17:41.316 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574857 2026-02-19T13:17:41.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:41.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574857 2026-02-19T13:17:41.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574857' 2026-02-19T13:17:41.318 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574857 2026-02-19T13:17:41.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:17:41.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574857 -lt 55834574857 2026-02-19T13:17:41.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:41.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378632 2026-02-19T13:17:41.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:41.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:17:41.543 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-81604378632 2026-02-19T13:17:41.543 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:41.544 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378632 2026-02-19T13:17:41.544 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 81604378632 2026-02-19T13:17:41.545 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 81604378632' 2026-02-19T13:17:41.545 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:17:41.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378632 -lt 81604378632 2026-02-19T13:17:41.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:41.764 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116999 2026-02-19T13:17:41.764 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:41.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:17:41.765 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964116999 2026-02-19T13:17:41.765 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:41.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116999 2026-02-19T13:17:41.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964116999' 2026-02-19T13:17:41.767 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 115964116999 2026-02-19T13:17:41.767 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:17:41.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964117000 -lt 115964116999 2026-02-19T13:17:41.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:41.990 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 4-150323855367 2026-02-19T13:17:41.990 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:41.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=4 2026-02-19T13:17:41.991 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 4-150323855367 2026-02-19T13:17:41.991 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:41.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=150323855367 2026-02-19T13:17:41.992 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.4 seq 150323855367 2026-02-19T13:17:41.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.4 seq 150323855367' 2026-02-19T13:17:41.993 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 4 2026-02-19T13:17:42.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 150323855368 -lt 150323855367 2026-02-19T13:17:42.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:42.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 5-184683593734 2026-02-19T13:17:42.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:42.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=5 2026-02-19T13:17:42.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 5-184683593734 2026-02-19T13:17:42.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:42.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593734 2026-02-19T13:17:42.219 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.5 seq 184683593734 2026-02-19T13:17:42.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.5 seq 184683593734' 2026-02-19T13:17:42.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 5 2026-02-19T13:17:42.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593735 -lt 184683593734 2026-02-19T13:17:42.442 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:17:42.442 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:17:42.442 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:17:42.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:17:42.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:17:42.744 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:17:42.744 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:17:42.744 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:17:42.744 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:17:42.744 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:17:42.744 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:17:42.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:17:42.960 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:17:42.960 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:17:42.960 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:17:43.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:17:43.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:17:43.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:17:43.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:431: TEST_ec_single_recovery_error: rados_get td/test-erasure-eio pool-jerasure myobject 2026-02-19T13:17:43.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:17:43.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:17:43.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=myobject 2026-02-19T13:17:43.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:17:43.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:17:43.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get myobject td/test-erasure-eio/COPY 2026-02-19T13:17:43.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:17:43.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:17:43.276 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:433: TEST_ec_single_recovery_error: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:17:43.277 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:17:43.277 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:17:43.542 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:17:43.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:17:43.820 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:17:43.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:17:43.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:17:43.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:17:43.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:17:43.831 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:43.831 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:43.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:43.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:43.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:43.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:43.940 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:17:43.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:17:43.941 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:17:43.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:17:43.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:17:43.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:17:43.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:43.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:17:43.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:17:43.944 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:43.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:17:43.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:17:43.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:17:43.959 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:17:43.959 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:43.959 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:43.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:17:43.961 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:17:43.961 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:17:43.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:17:43.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:17:43.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:17:43.963 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:17:43.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:17:43.964 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:17:43.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:17:43.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:17:43.966 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:17:43.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:43.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:17:43.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:17:43.967 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:17:43.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:17:43.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:17:43.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:17:43.970 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:17:43.970 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:43.970 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:43.970 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:17:43.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:17:43.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:17:43.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:17:43.972 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:17:43.972 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:43.972 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:43.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:17:43.973 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:17:43.975 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:17:43.995 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:17:43.995 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:43.995 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:43.995 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:43.995 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:43.995 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:43.995 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:43.996 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:17:44.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:17:44.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:17:44.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:17:44.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:17:44.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:17:44.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:17:44.020 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:17:44.080 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:17:44.080 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:17:44.080 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:44.081 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:17:44.082 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:17:44.082 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:17:44.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:17:44.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:17:44.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:17:44.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:17:44.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:17:44.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:17:44.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:17:44.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:17:44.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:44.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:44.252 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:44.252 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:44.253 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:44.253 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:44.253 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:17:44.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:17:44.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:17:44.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:17:44.371 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:17:44.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:17:45.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:17:45.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:17:45.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:17:45.384 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:45.384 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:45.384 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:45.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:17:45.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:17:45.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:17:45.440 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:17:45.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:17:45.448 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:17:44.012+0000 7fc4452c4dc0 0 load: jerasure load: lrc 2026-02-19T13:17:45.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_0 td/test-erasure-eio 2026-02-19T13:17:45.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:322: TEST_rados_get_bad_size_shard_0: local dir=td/test-erasure-eio 2026-02-19T13:17:45.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:323: TEST_rados_get_bad_size_shard_0: setup_osds 4 2026-02-19T13:17:45.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T13:17:45.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:17:45.450 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T13:17:45.457 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T13:17:45.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:45.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:17:45.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:45.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:45.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:17:45.458 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:45.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:45.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:45.459 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:45.459 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:45.459 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:45.459 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:45.459 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:45.459 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:45.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:45.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:45.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:45.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:17:45.461 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:45.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=eb68d5c5-2e89-4467-8dcc-1cdfac2e8643 2026-02-19T13:17:45.461 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 eb68d5c5-2e89-4467-8dcc-1cdfac2e8643 2026-02-19T13:17:45.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 eb68d5c5-2e89-4467-8dcc-1cdfac2e8643' 2026-02-19T13:17:45.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:45.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB5DZdpSI2+GxAAtUhEcor1ZAgrAKvcEPCvRA== 2026-02-19T13:17:45.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB5DZdpSI2+GxAAtUhEcor1ZAgrAKvcEPCvRA=="}' 2026-02-19T13:17:45.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new eb68d5c5-2e89-4467-8dcc-1cdfac2e8643 -i td/test-erasure-eio/0/new.json 2026-02-19T13:17:45.683 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:45.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:17:45.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB5DZdpSI2+GxAAtUhEcor1ZAgrAKvcEPCvRA== --osd-uuid eb68d5c5-2e89-4467-8dcc-1cdfac2e8643 2026-02-19T13:17:45.710 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:45.711+0000 7fea2347a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:45.712 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:45.713+0000 7fea2347a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:45.713 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:45.714+0000 7fea2347a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:45.713 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:45.714+0000 7fea2347a900 -1 bdev(0x55c1ef2f8c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:45.713 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:45.714+0000 7fea2347a900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:17:46.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:17:46.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:46.156 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:17:46.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:17:46.157 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:46.474 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:17:46.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:17:46.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:46.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:46.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:46.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:46.489 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:46.489+0000 7f829b178900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:46.490 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:46.491+0000 7f829b178900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:46.491 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:46.492+0000 7f829b178900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:46.688 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:46.689+0000 7f829b178900 -1 Falling back to public interface 2026-02-19T13:17:46.693 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:17:46.693 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:46.693 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:17:46.693 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:46.693 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:46.693 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:46.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:46.694 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:46.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:46.694 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:17:46.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:47.009 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:47.009+0000 7f829b178900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:17:47.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:47.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:47.919 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:47.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:47.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:47.919 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:17:48.146 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3131961999,v1:127.0.0.1:6803/3131961999] [v2:127.0.0.1:6804/3131961999,v1:127.0.0.1:6805/3131961999] exists,up eb68d5c5-2e89-4467-8dcc-1cdfac2e8643 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:48.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:48.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:48.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=59ddcbff-7288-49cd-986c-5fdddf66c24d 2026-02-19T13:17:48.152 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 59ddcbff-7288-49cd-986c-5fdddf66c24d 2026-02-19T13:17:48.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 59ddcbff-7288-49cd-986c-5fdddf66c24d' 2026-02-19T13:17:48.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:48.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB8DZdprdSeCRAA2QITwwLgV/U9XzPD/2Zv0g== 2026-02-19T13:17:48.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB8DZdprdSeCRAA2QITwwLgV/U9XzPD/2Zv0g=="}' 2026-02-19T13:17:48.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 59ddcbff-7288-49cd-986c-5fdddf66c24d -i td/test-erasure-eio/1/new.json 2026-02-19T13:17:48.379 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:48.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:17:48.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB8DZdprdSeCRAA2QITwwLgV/U9XzPD/2Zv0g== --osd-uuid 59ddcbff-7288-49cd-986c-5fdddf66c24d 2026-02-19T13:17:48.408 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:48.408+0000 7fb823389900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:48.409 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:48.410+0000 7fb823389900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:48.410 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:48.410+0000 7fb823389900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:48.411 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:48.411+0000 7fb823389900 -1 bdev(0x5611edd16c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:48.411 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:48.411+0000 7fb823389900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:17:48.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:17:48.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:48.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:17:48.835 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:17:48.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:49.139 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:17:49.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:17:49.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:49.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:49.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:49.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:49.155 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:49.155+0000 7fa0dc408900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:49.156 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:49.156+0000 7fa0dc408900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:49.157 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:49.157+0000 7fa0dc408900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:49.356 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:49.357+0000 7fa0dc408900 -1 Falling back to public interface 2026-02-19T13:17:49.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:17:49.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:49.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:17:49.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:49.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:49.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:49.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:49.361 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:49.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:49.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:17:49.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:49.690 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:49.690+0000 7fa0dc408900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:17:50.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:50.588 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:50.589 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:50.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:50.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:50.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:17:50.812 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2486255287,v1:127.0.0.1:6811/2486255287] [v2:127.0.0.1:6812/2486255287,v1:127.0.0.1:6813/2486255287] exists,up 59ddcbff-7288-49cd-986c-5fdddf66c24d 2026-02-19T13:17:50.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:50.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:50.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:50.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:50.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:17:50.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:50.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:50.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:50.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1a8d3d13-56de-405d-880e-c496f90ae578 2026-02-19T13:17:50.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 1a8d3d13-56de-405d-880e-c496f90ae578' 2026-02-19T13:17:50.820 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 1a8d3d13-56de-405d-880e-c496f90ae578 2026-02-19T13:17:50.820 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:50.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQB+DZdptBxgMRAAl096MoEMeoLqdCDcd3EjUQ== 2026-02-19T13:17:50.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQB+DZdptBxgMRAAl096MoEMeoLqdCDcd3EjUQ=="}' 2026-02-19T13:17:50.828 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1a8d3d13-56de-405d-880e-c496f90ae578 -i td/test-erasure-eio/2/new.json 2026-02-19T13:17:51.045 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:17:51.056 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:17:51.057 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQB+DZdptBxgMRAAl096MoEMeoLqdCDcd3EjUQ== --osd-uuid 1a8d3d13-56de-405d-880e-c496f90ae578 2026-02-19T13:17:51.073 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:51.073+0000 7f1b9d8e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:51.074 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:51.074+0000 7f1b9d8e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:51.075 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:51.075+0000 7f1b9d8e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:51.076 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:51.076+0000 7f1b9d8e4900 -1 bdev(0x55c7512b6c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:51.076 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:51.076+0000 7f1b9d8e4900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:17:51.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:17:51.523 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:51.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:17:51.524 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:17:51.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:51.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:17:51.826 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:17:51.827 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:51.827 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:51.827 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:51.827 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:51.840 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:51.840+0000 7f69a47e2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:51.841 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:51.841+0000 7f69a47e2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:51.842 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:51.842+0000 7f69a47e2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:52.038 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:52.038+0000 7f69a47e2900 -1 Falling back to public interface 2026-02-19T13:17:52.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:17:52.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:52.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:17:52.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:52.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:52.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:52.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:52.045 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:52.045 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:52.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:17:52.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:52.431 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:52.432+0000 7f69a47e2900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:17:53.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:53.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:53.268 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:53.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:53.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:53.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:17:53.492 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 17 up_thru 17 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1881487268,v1:127.0.0.1:6819/1881487268] [v2:127.0.0.1:6820/1881487268,v1:127.0.0.1:6821/1881487268] exists,up 1a8d3d13-56de-405d-880e-c496f90ae578 2026-02-19T13:17:53.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:53.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:17:53.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:53.498 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:17:53.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:17:53.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:17:53.500 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:17:53.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=96555ff6-fcc6-4689-876d-7230aaacfe00 2026-02-19T13:17:53.500 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 96555ff6-fcc6-4689-876d-7230aaacfe00 2026-02-19T13:17:53.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 96555ff6-fcc6-4689-876d-7230aaacfe00' 2026-02-19T13:17:53.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:17:53.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQCBDZdpb89GHhAAsImwHz69ajU2Cuy+XAJ1AQ== 2026-02-19T13:17:53.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQCBDZdpb89GHhAAsImwHz69ajU2Cuy+XAJ1AQ=="}' 2026-02-19T13:17:53.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 96555ff6-fcc6-4689-876d-7230aaacfe00 -i td/test-erasure-eio/3/new.json 2026-02-19T13:17:53.724 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:17:53.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:17:53.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQCBDZdpb89GHhAAsImwHz69ajU2Cuy+XAJ1AQ== --osd-uuid 96555ff6-fcc6-4689-876d-7230aaacfe00 2026-02-19T13:17:53.752 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:53.752+0000 7f006be4e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:53.753 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:53.753+0000 7f006be4e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:53.754 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:53.754+0000 7f006be4e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:53.754 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:53.755+0000 7f006be4e900 -1 bdev(0x55971d1fac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:17:53.754 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:53.755+0000 7f006be4e900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:17:54.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:17:54.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:17:54.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:17:54.147 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:17:54.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:17:54.446 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:17:54.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:17:54.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:17:54.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:17:54.447 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:17:54.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:17:54.461 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:54.461+0000 7f84c57df900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:54.462 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:54.462+0000 7f84c57df900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:54.463 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:54.463+0000 7f84c57df900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:17:54.657 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:54.657+0000 7f84c57df900 -1 Falling back to public interface 2026-02-19T13:17:54.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:17:54.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:17:54.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:17:54.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:17:54.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:17:54.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:54.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:17:54.668 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:17:54.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:54.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:17:54.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:17:54.996 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:17:54.996+0000 7f84c57df900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:17:55.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:17:55.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:17:55.887 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:17:55.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:17:55.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:17:55.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:17:56.107 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 25 up_thru 26 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1113537770,v1:127.0.0.1:6827/1113537770] [v2:127.0.0.1:6828/1113537770,v1:127.0.0.1:6829/1113537770] exists,up 96555ff6-fcc6-4689-876d-7230aaacfe00 2026-02-19T13:17:56.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:17:56.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:17:56.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:17:56.167 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:17:56.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:17:56.177 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:17:46.691+0000 7f829b178900 0 load: jerasure load: lrc 2026-02-19T13:17:56.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:325: TEST_rados_get_bad_size_shard_0: local poolname=pool-jerasure 2026-02-19T13:17:56.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:326: TEST_rados_get_bad_size_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T13:17:56.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:17:56.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:17:56.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T13:17:56.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:17:56.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T13:17:56.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:17:56.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T13:17:56.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:17:56.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:17:56.817 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:17:56.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:17:57.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:17:57.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:17:57.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:17:57.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:17:57.832 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:17:57.832 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:17:57.833 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:17:57.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:17:57.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:17:57.833 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:17:57.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:17:57.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:17:57.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:17:57.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:17:57.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:17:57.913 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:17:58.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:17:58.136 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:17:58.136 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:17:58.136 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:17:58.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:17:58.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:58.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:17:58.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803782 2026-02-19T13:17:58.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803782 2026-02-19T13:17:58.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782' 2026-02-19T13:17:58.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:58.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:17:58.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-19T13:17:58.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-19T13:17:58.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607556' 2026-02-19T13:17:58.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:58.295 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:17:58.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444035 2026-02-19T13:17:58.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444035 2026-02-19T13:17:58.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607556 2-73014444035' 2026-02-19T13:17:58.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:17:58.372 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:17:58.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=107374182403 2026-02-19T13:17:58.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 107374182403 2026-02-19T13:17:58.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803782 1-51539607556 2-73014444035 3-107374182403' 2026-02-19T13:17:58.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:58.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803782 2026-02-19T13:17:58.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:58.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:17:58.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803782 2026-02-19T13:17:58.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:58.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803782 2026-02-19T13:17:58.453 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803782' 2026-02-19T13:17:58.453 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803782 2026-02-19T13:17:58.453 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:58.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803779 -lt 25769803782 2026-02-19T13:17:58.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:17:59.676 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:17:59.677 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:17:59.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803782 2026-02-19T13:17:59.896 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:17:59.897 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-19T13:17:59.897 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:17:59.898 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:17:59.899 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607556 2026-02-19T13:17:59.899 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:17:59.899 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607556 2026-02-19T13:17:59.900 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 51539607556 2026-02-19T13:17:59.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-19T13:17:59.900 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:00.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-19T13:18:00.126 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:00.126 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444035 2026-02-19T13:18:00.126 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:00.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:00.127 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444035 2026-02-19T13:18:00.128 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:00.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444035 2026-02-19T13:18:00.129 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 73014444035 2026-02-19T13:18:00.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444035' 2026-02-19T13:18:00.129 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:00.349 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444036 -lt 73014444035 2026-02-19T13:18:00.350 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:00.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-107374182403 2026-02-19T13:18:00.350 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:00.351 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:00.352 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-107374182403 2026-02-19T13:18:00.352 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:00.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=107374182403 2026-02-19T13:18:00.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 107374182403' 2026-02-19T13:18:00.353 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 107374182403 2026-02-19T13:18:00.354 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:00.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 107374182403 -lt 107374182403 2026-02-19T13:18:00.573 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:00.573 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:00.573 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:00.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:00.867 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:00.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:00.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:00.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:00.867 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:00.868 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:00.868 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:01.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:01.084 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:01.085 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:01.085 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:01.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:01.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:01.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:01.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:328: TEST_rados_get_bad_size_shard_0: local shard_id=0 2026-02-19T13:18:01.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:329: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 10 2026-02-19T13:18:01.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T13:18:01.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T13:18:01.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-19T13:18:01.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T13:18:01.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-19T13:18:01.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T13:18:01.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-19T13:18:01.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T13:18:01.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-65583-0-10 2026-02-19T13:18:01.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-65583-0-10 2026-02-19T13:18:01.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:18:01.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:18:01.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-65583-0-10 2026-02-19T13:18:01.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:01.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:18:01.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:01.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:18:01.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:01.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:18:01.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:01.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:18:01.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-65583-0-10 td/test-erasure-eio/ORIGINAL 2026-02-19T13:18:01.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-65583-0-10 td/test-erasure-eio 0 10 set 2026-02-19T13:18:01.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-0-10 2026-02-19T13:18:01.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:18:01.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:18:01.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:18:01.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-19T13:18:01.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:18:01.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-19T13:18:01.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:18:01.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T13:18:01.408 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:18:01.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-0-10 2026-02-19T13:18:01.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:18:01.409 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-0-10 2026-02-19T13:18:01.409 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-0-10 2026-02-19T13:18:01.409 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:18:01.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:18:01.628 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:01.628 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:18:01.629 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:18:01.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:18:01.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:18:01.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-19T13:18:01.629 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:18:01.914 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:18:01.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T13:18:01.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-19T13:18:01.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-19T13:18:01.928 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records in 2026-02-19T13:18:01.928 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records out 2026-02-19T13:18:01.928 INFO:tasks.workunit.client.0.trial111.stderr:10 bytes copied, 1.8845e-05 s, 531 kB/s 2026-02-19T13:18:01.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-65583-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:01.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:18:01.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:18:01.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T13:18:01.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-65583-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:01.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:02.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:02.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-65583-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:02.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:18:02.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:18:02.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T13:18:02.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:18:02.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:02.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-65583-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:02.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-19T13:18:02.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:02.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:18:02.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T13:18:02.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:18:02.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:02.413 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T13:18:02.414 INFO:tasks.workunit.client.0.trial111.stderr:start osd.3 2026-02-19T13:18:02.415 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:02.415 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T13:18:02.415 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T13:18:02.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:18:02.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:18:02.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:18:02.429 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:02.429+0000 7fb133c14900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:02.430 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:02.431+0000 7fb133c14900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:02.431 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:02.431+0000 7fb133c14900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:02.594 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:02.594+0000 7fb133c14900 -1 Falling back to public interface 2026-02-19T13:18:02.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T13:18:02.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:02.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:18:02.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:02.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:02.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:02.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:02.640 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:18:02.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:02.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:18:02.859 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:02.969 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:02.969+0000 7fb133c14900 -1 osd.3 35 log_to_monitors true 2026-02-19T13:18:03.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:03.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:03.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:03.862 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:03.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:03.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:18:04.083 INFO:tasks.workunit.client.0.trial111.stderr:osd.3 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [25,35) [v2:127.0.0.1:6826/850685931,v1:127.0.0.1:6827/850685931] [v2:127.0.0.1:6828/850685931,v1:127.0.0.1:6829/850685931] exists,up 96555ff6-fcc6-4689-876d-7230aaacfe00 2026-02-19T13:18:04.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:04.084 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:04.085 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:04.085 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:04.085 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:04.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:04.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:04.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:04.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:04.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:04.164 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:04.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:04.383 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:04.383 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:18:04.383 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:18:04.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:04.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:04.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:04.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803785 2026-02-19T13:18:04.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803785 2026-02-19T13:18:04.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785' 2026-02-19T13:18:04.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:04.463 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:04.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607559 2026-02-19T13:18:04.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607559 2026-02-19T13:18:04.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607559' 2026-02-19T13:18:04.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:04.542 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:04.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444039 2026-02-19T13:18:04.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444039 2026-02-19T13:18:04.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607559 2-73014444039' 2026-02-19T13:18:04.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:04.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:18:04.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-19T13:18:04.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-19T13:18:04.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803785 1-51539607559 2-73014444039 3-167503724547' 2026-02-19T13:18:04.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:04.702 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803785 2026-02-19T13:18:04.703 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:04.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:04.703 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803785 2026-02-19T13:18:04.704 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:04.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803785 2026-02-19T13:18:04.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803785' 2026-02-19T13:18:04.704 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803785 2026-02-19T13:18:04.705 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:04.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803785 2026-02-19T13:18:04.924 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:04.924 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607559 2026-02-19T13:18:04.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:04.925 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:04.926 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607559 2026-02-19T13:18:04.926 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:04.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607559 2026-02-19T13:18:04.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607559' 2026-02-19T13:18:04.927 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 51539607559 2026-02-19T13:18:04.928 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:05.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607559 -lt 51539607559 2026-02-19T13:18:05.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:05.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444039 2026-02-19T13:18:05.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:05.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:05.153 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444039 2026-02-19T13:18:05.153 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:05.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444039 2026-02-19T13:18:05.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444039' 2026-02-19T13:18:05.154 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 73014444039 2026-02-19T13:18:05.154 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:05.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444039 -lt 73014444039 2026-02-19T13:18:05.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:05.382 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-167503724547 2026-02-19T13:18:05.382 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:05.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:05.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-167503724547 2026-02-19T13:18:05.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:05.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724547 2026-02-19T13:18:05.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 167503724547' 2026-02-19T13:18:05.384 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 167503724547 2026-02-19T13:18:05.385 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:05.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-19T13:18:05.611 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:05.611 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:05.611 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:05.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:05.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:05.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:05.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:05.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:05.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:05.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:05.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:06.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:06.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:06.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:06.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:06.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:06.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:06.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:06.429 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:18:06.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:18:06.746 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:18:06.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-0-10 2026-02-19T13:18:06.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:18:06.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:18:06.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-0-10 2026-02-19T13:18:06.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:18:06.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:18:06.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-0-10 td/test-erasure-eio/COPY 2026-02-19T13:18:06.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:18:06.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:18:06.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-65583-0-10 td/test-erasure-eio 1 10 set 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-0-10 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-19T13:18:06.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:18:06.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T13:18:06.783 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:18:06.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-0-10 2026-02-19T13:18:06.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:18:06.783 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-0-10 2026-02-19T13:18:06.783 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-0-10 2026-02-19T13:18:06.783 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:18:07.003 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:18:07.004 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:07.004 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:18:07.004 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:18:07.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:18:07.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:18:07.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T13:18:07.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:18:07.315 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:18:07.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T13:18:07.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-19T13:18:07.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-19T13:18:07.329 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records in 2026-02-19T13:18:07.329 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records out 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:10 bytes copied, 2.4766e-05 s, 404 kB/s 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-65583-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-65583-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:18:07.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:18:07.331 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:07.331 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:07.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:07.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:07.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:07.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:07.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-65583-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:07.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:18:07.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:18:07.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:18:07.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:18:07.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:07.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-65583-0-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:18:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:18:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:18:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:18:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:07.813 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:18:07.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:18:07.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:18:07.815 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:18:07.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:07.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:18:07.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:18:07.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:18:07.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:18:07.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:18:07.826 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:07.826+0000 7fd76f0b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:07.827 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:07.828+0000 7fd76f0b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:07.828 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:07.829+0000 7fd76f0b8900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:07.993 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:07.994+0000 7fd76f0b8900 -1 Falling back to public interface 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:18:08.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:08.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:08.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:08.360 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:08.360+0000 7fd76f0b8900 -1 osd.1 43 log_to_monitors true 2026-02-19T13:18:09.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:09.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:09.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:09.253 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:09.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:09.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:09.477 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 49 up_thru 49 down_at 46 last_clean_interval [12,43) [v2:127.0.0.1:6810/1637848243,v1:127.0.0.1:6811/1637848243] [v2:127.0.0.1:6812/1637848243,v1:127.0.0.1:6813/1637848243] exists,up 59ddcbff-7288-49cd-986c-5fdddf66c24d 2026-02-19T13:18:09.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:09.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:09.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:09.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:18:09.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:09.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:09.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:09.478 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:09.478 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:09.478 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:09.478 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:09.478 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:09.478 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:09.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:09.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:09.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:09.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:09.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:09.556 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:09.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:09.780 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:09.780 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:18:09.780 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:18:09.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:09.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:09.781 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:09.859 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803788 2026-02-19T13:18:09.859 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803788 2026-02-19T13:18:09.859 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803788' 2026-02-19T13:18:09.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:09.860 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:09.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-19T13:18:09.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-19T13:18:09.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803788 1-210453397507' 2026-02-19T13:18:09.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:09.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:10.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444042 2026-02-19T13:18:10.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444042 2026-02-19T13:18:10.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-73014444042' 2026-02-19T13:18:10.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:10.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:18:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724550 2026-02-19T13:18:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724550 2026-02-19T13:18:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803788 1-210453397507 2-73014444042 3-167503724550' 2026-02-19T13:18:10.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:10.103 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803788 2026-02-19T13:18:10.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:10.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:10.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803788 2026-02-19T13:18:10.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:10.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803788 2026-02-19T13:18:10.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803788' 2026-02-19T13:18:10.106 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803788 2026-02-19T13:18:10.106 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:10.329 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803786 -lt 25769803788 2026-02-19T13:18:10.329 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:11.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:11.330 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:11.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803788 -lt 25769803788 2026-02-19T13:18:11.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:11.555 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397507 2026-02-19T13:18:11.556 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:11.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:11.557 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-210453397507 2026-02-19T13:18:11.557 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:11.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397507 2026-02-19T13:18:11.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 210453397507' 2026-02-19T13:18:11.558 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 210453397507 2026-02-19T13:18:11.558 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:11.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-19T13:18:11.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:11.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444042 2026-02-19T13:18:11.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:11.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:11.780 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444042 2026-02-19T13:18:11.780 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:11.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444042 2026-02-19T13:18:11.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444042' 2026-02-19T13:18:11.782 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 73014444042 2026-02-19T13:18:11.782 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:12.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444042 -lt 73014444042 2026-02-19T13:18:12.004 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:12.005 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-167503724550 2026-02-19T13:18:12.005 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:12.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:12.007 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-167503724550 2026-02-19T13:18:12.007 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:12.008 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724550 2026-02-19T13:18:12.008 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 167503724550' 2026-02-19T13:18:12.008 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 167503724550 2026-02-19T13:18:12.008 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:12.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-19T13:18:12.226 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:12.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:12.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:12.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:12.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:12.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:12.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:12.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:12.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:12.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:12.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:12.741 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:12.741 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:12.741 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:12.742 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:13.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:13.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:13.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:13.038 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:18:13.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:18:13.352 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:18:13.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-19T13:18:13.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-0-10 fail 2026-02-19T13:18:13.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:18:13.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:18:13.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-0-10 2026-02-19T13:18:13.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-19T13:18:13.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-19T13:18:13.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-65583-0-10 td/test-erasure-eio/COPY 2026-02-19T13:18:13.381 INFO:tasks.workunit.client.0.trial111.stderr:error getting pool-jerasure/obj-size-65583-0-10: (5) Input/output error 2026-02-19T13:18:13.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-19T13:18:13.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:330: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 0 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-19T13:18:13.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-65583-0-0 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-65583-0-0 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-65583-0-0 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:18:13.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:13.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:18:13.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-65583-0-0 td/test-erasure-eio/ORIGINAL 2026-02-19T13:18:13.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-65583-0-0 td/test-erasure-eio 0 0 set 2026-02-19T13:18:13.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-0-0 2026-02-19T13:18:13.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:18:13.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:18:13.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:18:13.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-19T13:18:13.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:18:13.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-19T13:18:13.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:18:13.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T13:18:13.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:18:13.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-0-0 2026-02-19T13:18:13.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:18:13.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-0-0 2026-02-19T13:18:13.406 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-0-0 2026-02-19T13:18:13.406 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:18:13.629 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:18:13.629 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:13.630 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:18:13.630 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:18:13.630 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:18:13.630 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:18:13.630 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-19T13:18:13.630 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:18:13.916 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:18:13.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T13:18:13.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-19T13:18:13.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-65583-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-65583-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T13:18:13.930 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:13.931 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:13.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:13.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:13.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:14.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:14.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-65583-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:14.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:18:14.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:18:14.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T13:18:14.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:18:14.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:14.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-65583-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:14.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-19T13:18:14.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:14.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:18:14.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T13:18:14.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:14.397 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:18:14.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:14.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:14.398 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:14.398 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:14.398 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:14.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:14.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T13:18:14.401 INFO:tasks.workunit.client.0.trial111.stderr:start osd.3 2026-02-19T13:18:14.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:14.402 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T13:18:14.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T13:18:14.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:18:14.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:18:14.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:18:14.412 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:14.412+0000 7f6902694900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:14.413 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:14.414+0000 7f6902694900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:14.414 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:14.415+0000 7f6902694900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:14.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T13:18:14.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:18:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:14.620 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:18:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:14.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:18:14.621 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:14.621+0000 7f6902694900 -1 Falling back to public interface 2026-02-19T13:18:14.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:14.995 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:14.995+0000 7f6902694900 -1 osd.3 54 log_to_monitors true 2026-02-19T13:18:15.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:15.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:15.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:15.842 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:15.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:15.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:18:16.065 INFO:tasks.workunit.client.0.trial111.stderr:osd.3 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [39,54) [v2:127.0.0.1:6826/3694575078,v1:127.0.0.1:6827/3694575078] [v2:127.0.0.1:6828/3694575078,v1:127.0.0.1:6829/3694575078] exists,up 96555ff6-fcc6-4689-876d-7230aaacfe00 2026-02-19T13:18:16.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:16.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:16.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:16.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:18:16.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:16.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:16.066 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:16.066 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:16.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:16.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:16.066 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:16.066 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:16.066 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:16.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:16.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:16.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:16.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:16.145 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:16.145 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:16.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:16.366 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:16.366 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:18:16.366 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:18:16.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:16.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:16.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:16.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803791 2026-02-19T13:18:16.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803791 2026-02-19T13:18:16.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791' 2026-02-19T13:18:16.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:16.449 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:16.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-19T13:18:16.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-19T13:18:16.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-210453397510' 2026-02-19T13:18:16.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:16.526 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:16.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444045 2026-02-19T13:18:16.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444045 2026-02-19T13:18:16.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-210453397510 2-73014444045' 2026-02-19T13:18:16.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:16.609 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:18:16.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070467 2026-02-19T13:18:16.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070467 2026-02-19T13:18:16.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803791 1-210453397510 2-73014444045 3-253403070467' 2026-02-19T13:18:16.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:16.687 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803791 2026-02-19T13:18:16.687 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:16.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:16.688 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803791 2026-02-19T13:18:16.689 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:16.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803791 2026-02-19T13:18:16.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803791' 2026-02-19T13:18:16.690 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803791 2026-02-19T13:18:16.690 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:16.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803791 -lt 25769803791 2026-02-19T13:18:16.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:16.911 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-210453397510 2026-02-19T13:18:16.911 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:16.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:16.913 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-210453397510 2026-02-19T13:18:16.913 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:16.914 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397510 2026-02-19T13:18:16.914 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 210453397510' 2026-02-19T13:18:16.914 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 210453397510 2026-02-19T13:18:16.915 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:17.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-19T13:18:17.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:17.136 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444045 2026-02-19T13:18:17.136 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:17.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:17.138 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444045 2026-02-19T13:18:17.138 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:17.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444045 2026-02-19T13:18:17.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444045' 2026-02-19T13:18:17.139 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 73014444045 2026-02-19T13:18:17.139 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:17.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444045 -lt 73014444045 2026-02-19T13:18:17.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:17.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-253403070467 2026-02-19T13:18:17.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:17.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:17.363 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-253403070467 2026-02-19T13:18:17.363 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:17.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070467 2026-02-19T13:18:17.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 253403070467' 2026-02-19T13:18:17.364 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 253403070467 2026-02-19T13:18:17.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:17.584 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-19T13:18:17.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:17.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:17.585 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:17.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:17.886 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:17.886 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:17.886 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:17.887 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:17.887 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:17.887 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:17.887 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:18.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:18.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:18.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:18.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:18.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:18.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:18.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:18.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:18:18.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:18:18.722 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:18:18.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-0-0 2026-02-19T13:18:18.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:18:18.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:18:18.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-0-0 2026-02-19T13:18:18.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:18:18.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:18:18.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-0-0 td/test-erasure-eio/COPY 2026-02-19T13:18:18.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:18:18.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:18:18.757 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-65583-0-0 td/test-erasure-eio 1 0 set 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-0-0 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-0-0 2026-02-19T13:18:18.759 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:18:18.760 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-0-0 2026-02-19T13:18:18.760 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-0-0 2026-02-19T13:18:18.760 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:18:18.986 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:18:18.986 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:18.986 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:18:18.986 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:18:18.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:18:18.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:18:18.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T13:18:18.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:18:19.298 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:18:19.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T13:18:19.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-19T13:18:19.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-65583-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-65583-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:19.314 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:19.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:19.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:19.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:19.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:19.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-65583-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:19.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:18:19.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:18:19.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:18:19.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:18:19.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:19.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-65583-0-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:19.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:18:19.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:19.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:18:19.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:18:19.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:18:19.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:19.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:19.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:19.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:18:19.780 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:18:19.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:19.781 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:18:19.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:18:19.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:18:19.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:18:19.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:18:19.795 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:19.796+0000 7f6d7b4b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:19.796 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:19.798+0000 7f6d7b4b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:19.797 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:19.798+0000 7f6d7b4b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:20.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:18:20.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:20.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:18:20.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:20.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:20.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:20.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:20.006 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:18:20.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:20.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:20.017 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:20.019+0000 7f6d7b4b0900 -1 Falling back to public interface 2026-02-19T13:18:20.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:20.405 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:20.406+0000 7f6d7b4b0900 -1 osd.1 64 log_to_monitors true 2026-02-19T13:18:21.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:21.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:21.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:21.228 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:21.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:21.229 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 68 up_thru 68 down_at 65 last_clean_interval [49,64) [v2:127.0.0.1:6810/3661067919,v1:127.0.0.1:6811/3661067919] [v2:127.0.0.1:6812/3661067919,v1:127.0.0.1:6813/3661067919] exists,up 59ddcbff-7288-49cd-986c-5fdddf66c24d 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:21.450 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:21.451 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:21.451 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:21.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:21.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:21.451 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:21.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:21.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:21.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:21.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:21.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:21.531 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:21.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:21.748 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:21.749 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:18:21.749 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:18:21.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:21.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:21.749 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:21.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803794 2026-02-19T13:18:21.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803794 2026-02-19T13:18:21.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794' 2026-02-19T13:18:21.831 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:21.831 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:21.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776131 2026-02-19T13:18:21.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776131 2026-02-19T13:18:21.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-292057776131' 2026-02-19T13:18:21.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:21.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:21.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444048 2026-02-19T13:18:21.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444048 2026-02-19T13:18:21.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-292057776131 2-73014444048' 2026-02-19T13:18:21.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:21.991 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:18:22.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070470 2026-02-19T13:18:22.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070470 2026-02-19T13:18:22.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803794 1-292057776131 2-73014444048 3-253403070470' 2026-02-19T13:18:22.072 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:22.072 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803794 2026-02-19T13:18:22.072 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:22.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:22.073 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803794 2026-02-19T13:18:22.073 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:22.074 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803794 2026-02-19T13:18:22.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803794' 2026-02-19T13:18:22.075 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803794 2026-02-19T13:18:22.075 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:22.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803792 -lt 25769803794 2026-02-19T13:18:22.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:23.296 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:23.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:23.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803794 -lt 25769803794 2026-02-19T13:18:23.521 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:23.522 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776131 2026-02-19T13:18:23.522 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:23.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:23.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-292057776131 2026-02-19T13:18:23.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:23.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776131 2026-02-19T13:18:23.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 292057776131' 2026-02-19T13:18:23.525 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 292057776131 2026-02-19T13:18:23.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:23.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-19T13:18:23.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:23.748 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444048 2026-02-19T13:18:23.748 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:23.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:23.749 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444048 2026-02-19T13:18:23.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:23.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444048 2026-02-19T13:18:23.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444048' 2026-02-19T13:18:23.751 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 73014444048 2026-02-19T13:18:23.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:23.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444048 -lt 73014444048 2026-02-19T13:18:23.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:23.971 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-253403070470 2026-02-19T13:18:23.971 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:23.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:23.973 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-253403070470 2026-02-19T13:18:23.973 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:23.974 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070470 2026-02-19T13:18:23.974 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 253403070470' 2026-02-19T13:18:23.974 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 253403070470 2026-02-19T13:18:23.974 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:24.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070470 -lt 253403070470 2026-02-19T13:18:24.198 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:24.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:24.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:24.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:24.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:24.500 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:24.500 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:24.500 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:24.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:24.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:24.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:24.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:24.719 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:24.719 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:24.719 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:25.015 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:25.015 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:25.015 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:25.015 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:18:25.016 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:18:25.328 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:18:25.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-19T13:18:25.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-0-0 fail 2026-02-19T13:18:25.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:18:25.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:18:25.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-0-0 2026-02-19T13:18:25.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-19T13:18:25.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-19T13:18:25.343 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-65583-0-0 td/test-erasure-eio/COPY 2026-02-19T13:18:25.360 INFO:tasks.workunit.client.0.trial111.stderr:error getting pool-jerasure/obj-size-65583-0-0: (5) Input/output error 2026-02-19T13:18:25.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-19T13:18:25.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:331: TEST_rados_get_bad_size_shard_0: rados_get_data_bad_size td/test-erasure-eio 0 256 add 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=0 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-65583-0-256 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-65583-0-256 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-65583-0-256 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:25.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:18:25.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:25.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:18:25.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:25.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:18:25.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:18:25.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:18:25.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-65583-0-256 td/test-erasure-eio/ORIGINAL 2026-02-19T13:18:25.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-65583-0-256 td/test-erasure-eio 0 256 add 2026-02-19T13:18:25.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-0-256 2026-02-19T13:18:25.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:18:25.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:18:25.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:18:25.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=0 2026-02-19T13:18:25.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:18:25.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-19T13:18:25.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:18:25.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-19T13:18:25.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:18:25.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-0-256 2026-02-19T13:18:25.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:18:25.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-0-256 2026-02-19T13:18:25.383 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-0-256 2026-02-19T13:18:25.383 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:18:25.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:18:25.607 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:25.607 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:18:25.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:18:25.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:18:25.607 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:18:25.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=3 2026-02-19T13:18:25.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:18:25.894 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-65583-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-65583-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T13:18:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:18:25.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T13:18:25.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:25.908 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:25.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:25.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:25.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:26.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:26.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-65583-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:26.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:18:26.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:18:26.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T13:18:26.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:18:26.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:26.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-65583-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:18:26.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:18:26.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:18:26.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T13:18:26.337 INFO:tasks.workunit.client.0.trial111.stderr:start osd.3 2026-02-19T13:18:26.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:26.337 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T13:18:26.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T13:18:26.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:18:26.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:18:26.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:18:26.352 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:26.353+0000 7f23e8244900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:26.354 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:26.354+0000 7f23e8244900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:26.355 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:26.355+0000 7f23e8244900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:26.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T13:18:26.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:26.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:18:26.558 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:26.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:26.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:26.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:26.559 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:18:26.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:26.559 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:18:26.567 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:26.567+0000 7f23e8244900 -1 Falling back to public interface 2026-02-19T13:18:26.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:26.936 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:26.936+0000 7f23e8244900 -1 osd.3 71 log_to_monitors true 2026-02-19T13:18:27.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:27.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:27.781 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:27.782 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:27.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:27.782 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:18:28.000 INFO:tasks.workunit.client.0.trial111.stderr:osd.3 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,71) [v2:127.0.0.1:6826/3975026395,v1:127.0.0.1:6827/3975026395] [v2:127.0.0.1:6828/3975026395,v1:127.0.0.1:6829/3975026395] exists,up 96555ff6-fcc6-4689-876d-7230aaacfe00 2026-02-19T13:18:28.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:28.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:28.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:28.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:18:28.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:28.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:28.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:28.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:28.001 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:28.001 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:28.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:28.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:28.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:28.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:28.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:28.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:28.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:28.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:28.082 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:28.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:28.301 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:28.301 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:18:28.301 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:18:28.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:28.302 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:28.302 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:28.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803798 2026-02-19T13:18:28.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803798 2026-02-19T13:18:28.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803798' 2026-02-19T13:18:28.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:28.379 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:28.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776134 2026-02-19T13:18:28.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776134 2026-02-19T13:18:28.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803798 1-292057776134' 2026-02-19T13:18:28.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:28.459 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:28.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444051 2026-02-19T13:18:28.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444051 2026-02-19T13:18:28.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803798 1-292057776134 2-73014444051' 2026-02-19T13:18:28.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:28.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:18:28.617 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481795 2026-02-19T13:18:28.617 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481795 2026-02-19T13:18:28.617 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803798 1-292057776134 2-73014444051 3-330712481795' 2026-02-19T13:18:28.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:28.618 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803798 2026-02-19T13:18:28.618 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:28.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:28.619 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803798 2026-02-19T13:18:28.620 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:28.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803798 2026-02-19T13:18:28.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803798' 2026-02-19T13:18:28.621 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803798 2026-02-19T13:18:28.621 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:28.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803798 -lt 25769803798 2026-02-19T13:18:28.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:28.841 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776134 2026-02-19T13:18:28.841 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:28.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:28.843 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-292057776134 2026-02-19T13:18:28.843 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:28.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776134 2026-02-19T13:18:28.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 292057776134' 2026-02-19T13:18:28.844 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 292057776134 2026-02-19T13:18:28.844 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:29.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-19T13:18:29.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:29.066 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444051 2026-02-19T13:18:29.066 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:29.067 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:29.067 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444051 2026-02-19T13:18:29.068 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:29.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444051 2026-02-19T13:18:29.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444051' 2026-02-19T13:18:29.069 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 73014444051 2026-02-19T13:18:29.069 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:29.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444052 -lt 73014444051 2026-02-19T13:18:29.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:29.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-330712481795 2026-02-19T13:18:29.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:29.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:29.296 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-330712481795 2026-02-19T13:18:29.296 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:29.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=330712481795 2026-02-19T13:18:29.297 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 330712481795' 2026-02-19T13:18:29.297 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 330712481795 2026-02-19T13:18:29.297 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:29.519 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-19T13:18:29.519 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:29.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:29.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:29.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:29.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:29.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:29.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:29.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:29.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:29.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:29.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:30.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:30.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:30.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:30.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:30.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:30.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:30.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:30.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records in 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records out 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:256 bytes copied, 1.8404e-05 s, 13.9 MB/s 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 3 obj-size-65583-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=3 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 3 obj-size-65583-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:30.337 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=3 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.3 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:30.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:30.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:30.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 3 obj-size-65583-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:30.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:18:30.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:18:30.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=3 2026-02-19T13:18:30.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:18:30.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:30.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/3 obj-size-65583-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 3 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=3 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:30.638 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:18:30.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.3 2026-02-19T13:18:30.640 INFO:tasks.workunit.client.0.trial111.stderr:start osd.3 2026-02-19T13:18:30.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:30.640 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/3/whoami 2026-02-19T13:18:30.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 3 = 3 ']' 2026-02-19T13:18:30.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:18:30.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:18:30.641 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:18:30.654 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:30.655+0000 7f3c7c8b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:30.656 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:30.656+0000 7f3c7c8b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:30.656 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:30.657+0000 7f3c7c8b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:30.845 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:30.845+0000 7f3c7c8b0900 -1 Falling back to public interface 2026-02-19T13:18:30.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 3 2026-02-19T13:18:30.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:30.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:18:30.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:30.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:30.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:30.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:30.864 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:18:30.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:30.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:18:31.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:31.243 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:31.243+0000 7f3c7c8b0900 -1 osd.3 79 log_to_monitors true 2026-02-19T13:18:32.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:32.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:32.083 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:32.083 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:32.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:32.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:18:32.307 INFO:tasks.workunit.client.0.trial111.stderr:osd.3 up in weight 1 up_from 83 up_thru 83 down_at 80 last_clean_interval [77,79) [v2:127.0.0.1:6826/3932231220,v1:127.0.0.1:6827/3932231220] [v2:127.0.0.1:6828/3932231220,v1:127.0.0.1:6829/3932231220] exists,up 96555ff6-fcc6-4689-876d-7230aaacfe00 2026-02-19T13:18:32.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:32.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:32.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:32.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:18:32.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:32.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:32.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:32.308 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:32.309 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:32.309 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:32.309 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:32.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:32.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:32.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:32.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:32.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:32.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:32.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:32.388 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:32.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:32.617 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:32.617 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:18:32.617 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:18:32.617 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:32.617 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:32.617 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:32.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803800 2026-02-19T13:18:32.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803800 2026-02-19T13:18:32.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800' 2026-02-19T13:18:32.695 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:32.695 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:32.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776137 2026-02-19T13:18:32.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776137 2026-02-19T13:18:32.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-292057776137' 2026-02-19T13:18:32.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:32.774 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:32.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444054 2026-02-19T13:18:32.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444054 2026-02-19T13:18:32.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-292057776137 2-73014444054' 2026-02-19T13:18:32.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:32.854 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:18:32.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285571 2026-02-19T13:18:32.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285571 2026-02-19T13:18:32.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-292057776137 2-73014444054 3-356482285571' 2026-02-19T13:18:32.934 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:32.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803800 2026-02-19T13:18:32.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:32.935 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:32.936 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803800 2026-02-19T13:18:32.936 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:32.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803800 2026-02-19T13:18:32.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803800' 2026-02-19T13:18:32.937 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803800 2026-02-19T13:18:32.937 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:33.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803800 -lt 25769803800 2026-02-19T13:18:33.159 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:33.159 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-292057776137 2026-02-19T13:18:33.159 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:33.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:33.161 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-292057776137 2026-02-19T13:18:33.161 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776137 2026-02-19T13:18:33.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 292057776137' 2026-02-19T13:18:33.162 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 292057776137 2026-02-19T13:18:33.163 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:33.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776135 -lt 292057776137 2026-02-19T13:18:33.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:34.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:34.387 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:34.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776135 -lt 292057776137 2026-02-19T13:18:34.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:35.611 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:18:35.611 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:35.837 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776137 -lt 292057776137 2026-02-19T13:18:35.838 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:35.838 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444054 2026-02-19T13:18:35.838 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:35.839 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:35.840 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444054 2026-02-19T13:18:35.840 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:35.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444054 2026-02-19T13:18:35.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444054' 2026-02-19T13:18:35.841 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 73014444054 2026-02-19T13:18:35.841 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:36.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444055 -lt 73014444054 2026-02-19T13:18:36.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:36.060 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285571 2026-02-19T13:18:36.060 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:36.061 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:36.062 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-356482285571 2026-02-19T13:18:36.062 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:36.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285571 2026-02-19T13:18:36.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 356482285571' 2026-02-19T13:18:36.063 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 356482285571 2026-02-19T13:18:36.063 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:36.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285571 -lt 356482285571 2026-02-19T13:18:36.283 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:36.283 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:36.283 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:36.578 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:36.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:36.579 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:36.579 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:36.579 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:36.579 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:36.579 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:36.579 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:36.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:36.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:36.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:36.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:37.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:37.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:37.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:37.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:18:37.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:18:37.419 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:18:37.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-0-256 2026-02-19T13:18:37.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:18:37.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:18:37.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-0-256 2026-02-19T13:18:37.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:18:37.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:18:37.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-0-256 td/test-erasure-eio/COPY 2026-02-19T13:18:37.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:18:37.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:18:37.453 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 0 + 1 2026-02-19T13:18:37.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=1 2026-02-19T13:18:37.455 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-65583-0-256 td/test-erasure-eio 1 256 add 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-0-256 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:18:37.456 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-19T13:18:37.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:18:37.457 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-0-256 2026-02-19T13:18:37.457 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:18:37.457 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-0-256 2026-02-19T13:18:37.457 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-0-256 2026-02-19T13:18:37.457 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:18:37.676 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:18:37.677 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:37.677 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:18:37.677 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:18:37.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:18:37.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:18:37.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T13:18:37.677 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:18:37.986 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:18:38.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-65583-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-65583-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:38.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:38.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:38.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-65583-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:38.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:18:38.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:18:38.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:18:38.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:18:38.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:38.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-65583-0-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:38.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:18:38.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:38.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:18:38.435 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:38.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:18:38.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:18:38.438 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:18:38.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:38.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:18:38.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:18:38.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:18:38.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:18:38.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:18:38.447 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:38.448+0000 7ff3ea814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:38.449 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:38.449+0000 7ff3ea814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:38.450 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:38.450+0000 7ff3ea814900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:38.608 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:38.608+0000 7ff3ea814900 -1 Falling back to public interface 2026-02-19T13:18:38.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:18:38.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:38.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:18:38.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:38.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:38.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:38.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:38.657 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:18:38.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:38.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:38.873 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:38.873+0000 7ff3ea814900 -1 osd.1 86 log_to_monitors true 2026-02-19T13:18:38.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:39.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:39.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:39.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:39.882 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:39.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:39.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 92 up_thru 92 down_at 89 last_clean_interval [68,86) [v2:127.0.0.1:6810/817114643,v1:127.0.0.1:6811/817114643] [v2:127.0.0.1:6812/817114643,v1:127.0.0.1:6813/817114643] exists,up 59ddcbff-7288-49cd-986c-5fdddf66c24d 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:40.104 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:40.105 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:40.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:40.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:40.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:40.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:40.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:40.185 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:40.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:40.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:40.186 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:40.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:40.403 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:40.403 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:18:40.403 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:18:40.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:40.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:40.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:40.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803804 2026-02-19T13:18:40.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803804 2026-02-19T13:18:40.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804' 2026-02-19T13:18:40.481 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:40.481 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:40.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=395136991235 2026-02-19T13:18:40.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 395136991235 2026-02-19T13:18:40.563 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-395136991235' 2026-02-19T13:18:40.564 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:40.564 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:40.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444058 2026-02-19T13:18:40.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444058 2026-02-19T13:18:40.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-395136991235 2-73014444058' 2026-02-19T13:18:40.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:40.644 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:18:40.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285574 2026-02-19T13:18:40.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285574 2026-02-19T13:18:40.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803804 1-395136991235 2-73014444058 3-356482285574' 2026-02-19T13:18:40.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:40.725 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803804 2026-02-19T13:18:40.725 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:40.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:40.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803804 2026-02-19T13:18:40.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:40.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803804 2026-02-19T13:18:40.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803804' 2026-02-19T13:18:40.727 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803804 2026-02-19T13:18:40.727 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:40.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803804 -lt 25769803804 2026-02-19T13:18:40.947 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:40.948 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-395136991235 2026-02-19T13:18:40.948 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:40.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:40.949 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-395136991235 2026-02-19T13:18:40.949 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:40.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=395136991235 2026-02-19T13:18:40.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 395136991235' 2026-02-19T13:18:40.951 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 395136991235 2026-02-19T13:18:40.951 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:41.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 395136991235 -lt 395136991235 2026-02-19T13:18:41.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:41.176 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444058 2026-02-19T13:18:41.176 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:41.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:41.177 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444058 2026-02-19T13:18:41.177 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:41.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444058 2026-02-19T13:18:41.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444058' 2026-02-19T13:18:41.179 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 73014444058 2026-02-19T13:18:41.179 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:41.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444058 -lt 73014444058 2026-02-19T13:18:41.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:41.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285574 2026-02-19T13:18:41.399 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:41.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:41.400 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-356482285574 2026-02-19T13:18:41.400 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:41.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285574 2026-02-19T13:18:41.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 356482285574' 2026-02-19T13:18:41.401 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 356482285574 2026-02-19T13:18:41.402 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:41.622 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285574 -lt 356482285574 2026-02-19T13:18:41.622 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:41.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:41.623 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:41.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:41.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:41.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:41.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:41.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:41.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:41.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:41.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:42.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:42.139 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:42.139 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:42.139 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:42.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:42.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:42.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:42.436 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records in 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records out 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:256 bytes copied, 1.8013e-05 s, 14.2 MB/s 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-65583-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-65583-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:42.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:42.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:42.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-65583-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:42.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:18:42.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:18:42.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:18:42.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:18:42.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:42.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-65583-0-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:18:42.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:42.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:42.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:18:42.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:18:42.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:18:42.945 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:18:42.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:42.945 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:18:42.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:18:42.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:18:42.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:18:42.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:18:42.953 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:42.953+0000 7f0850279900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:42.954 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:42.954+0000 7f0850279900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:42.955 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:42.955+0000 7f0850279900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:43.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:43.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:43.195 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:43.196+0000 7f0850279900 -1 Falling back to public interface 2026-02-19T13:18:43.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:43.544 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:43.544+0000 7f0850279900 -1 osd.1 93 log_to_monitors true 2026-02-19T13:18:44.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:44.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:44.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:44.390 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:44.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:44.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 97 up_thru 97 down_at 94 last_clean_interval [92,93) [v2:127.0.0.1:6810/1591229365,v1:127.0.0.1:6811/1591229365] [v2:127.0.0.1:6812/1591229365,v1:127.0.0.1:6813/1591229365] exists,up 59ddcbff-7288-49cd-986c-5fdddf66c24d 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:18:44.609 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:18:44.610 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:18:44.610 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:18:44.610 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:18:44.610 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:18:44.610 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:18:44.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:18:44.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:18:44.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:18:44.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:18:44.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:18:44.690 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:18:44.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:18:44.909 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:18:44.909 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:18:44.909 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:18:44.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:18:44.909 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:44.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:18:44.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803807 2026-02-19T13:18:44.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803807 2026-02-19T13:18:44.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807' 2026-02-19T13:18:44.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:44.991 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:18:45.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=416611827715 2026-02-19T13:18:45.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 416611827715 2026-02-19T13:18:45.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-416611827715' 2026-02-19T13:18:45.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:45.071 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:18:45.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=73014444061 2026-02-19T13:18:45.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 73014444061 2026-02-19T13:18:45.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-416611827715 2-73014444061' 2026-02-19T13:18:45.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:18:45.149 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:18:45.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=356482285577 2026-02-19T13:18:45.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 356482285577 2026-02-19T13:18:45.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803807 1-416611827715 2-73014444061 3-356482285577' 2026-02-19T13:18:45.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:45.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803807 2026-02-19T13:18:45.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:45.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:18:45.229 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803807 2026-02-19T13:18:45.229 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:45.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803807 2026-02-19T13:18:45.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803807' 2026-02-19T13:18:45.230 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803807 2026-02-19T13:18:45.231 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:45.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-19T13:18:45.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:46.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:18:46.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:46.673 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803805 -lt 25769803807 2026-02-19T13:18:46.674 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:18:47.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:18:47.675 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:18:47.898 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803807 -lt 25769803807 2026-02-19T13:18:47.898 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:47.899 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-416611827715 2026-02-19T13:18:47.899 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:47.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:18:47.901 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-416611827715 2026-02-19T13:18:47.901 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:47.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=416611827715 2026-02-19T13:18:47.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 416611827715' 2026-02-19T13:18:47.902 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 416611827715 2026-02-19T13:18:47.902 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:18:48.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 416611827715 -lt 416611827715 2026-02-19T13:18:48.120 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:48.121 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-73014444061 2026-02-19T13:18:48.121 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:48.121 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:18:48.122 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-73014444061 2026-02-19T13:18:48.122 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:48.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=73014444061 2026-02-19T13:18:48.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 73014444061' 2026-02-19T13:18:48.123 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 73014444061 2026-02-19T13:18:48.124 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:18:48.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 73014444061 -lt 73014444061 2026-02-19T13:18:48.344 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:18:48.345 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-356482285577 2026-02-19T13:18:48.345 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:18:48.346 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:18:48.346 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-356482285577 2026-02-19T13:18:48.346 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:18:48.347 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=356482285577 2026-02-19T13:18:48.348 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 356482285577' 2026-02-19T13:18:48.348 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 356482285577 2026-02-19T13:18:48.348 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:18:48.571 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 356482285578 -lt 356482285577 2026-02-19T13:18:48.572 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:18:48.572 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:48.573 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:48.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:18:48.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:18:48.869 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:18:48.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:18:48.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:18:48.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:18:48.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:18:48.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:18:49.086 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:18:49.086 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:18:49.086 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:18:49.086 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:18:49.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:18:49.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:18:49.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:18:49.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:18:49.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:18:49.699 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:18:49.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-19T13:18:49.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:262: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-0-256 fail 2026-02-19T13:18:49.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:18:49.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:18:49.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-0-256 2026-02-19T13:18:49.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-19T13:18:49.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-19T13:18:49.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-size-65583-0-256 td/test-erasure-eio/COPY 2026-02-19T13:18:49.730 INFO:tasks.workunit.client.0.trial111.stderr:error getting pool-jerasure/obj-size-65583-0-256: (5) Input/output error 2026-02-19T13:18:49.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-19T13:18:49.732 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:18:49.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:332: TEST_rados_get_bad_size_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:18:49.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:18:49.733 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:18:49.955 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:18:49.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:18:50.239 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:18:50.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:18:50.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:18:50.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:18:50.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:18:50.253 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:50.253 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:50.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:50.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:50.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:50.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:50.360 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:18:50.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:18:50.361 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:18:50.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:18:50.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:18:50.362 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:18:50.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:18:50.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:18:50.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:18:50.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:18:50.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:18:50.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:18:50.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:18:50.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:18:50.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:50.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:50.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:18:50.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:18:50.386 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:18:50.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:18:50.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:18:50.387 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:18:50.387 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:18:50.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:18:50.388 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:18:50.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:18:50.389 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:18:50.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:18:50.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:18:50.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:18:50.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:18:50.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:18:50.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:18:50.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:18:50.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:18:50.393 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:18:50.394 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:50.394 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:50.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:18:50.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:18:50.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:18:50.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:18:50.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:18:50.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:50.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:50.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:18:50.397 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:18:50.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:18:50.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:18:50.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:18:50.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:18:50.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:18:50.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:18:50.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:18:50.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:18:50.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:18:50.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:18:50.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:18:50.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:50.418 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:50.418 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:50.419 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:50.419 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:50.419 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:50.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:18:50.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:18:50.443 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:18:50.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:18:50.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:18:50.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:18:50.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:18:50.445 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:18:50.445 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:18:50.445 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:18:50.445 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:18:50.445 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:50.445 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:50.445 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:18:50.446 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:18:50.446 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:18:50.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:18:50.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:18:50.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:18:50.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:18:50.506 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:18:50.506 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:18:50.506 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:18:50.506 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:18:50.507 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:18:50.507 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:50.508 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:50.508 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:18:50.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:18:50.508 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:18:50.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:18:50.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:18:50.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:18:50.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:18:50.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:18:50.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:18:50.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:18:50.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:18:50.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:50.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:50.679 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:50.679 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:50.679 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:50.679 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:50.680 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:18:50.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:18:50.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:18:50.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:18:50.800 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:18:50.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:18:51.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:18:51.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:18:51.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:18:51.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:18:51.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:51.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:51.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:18:51.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:18:51.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:18:51.869 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:18:51.876 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:18:51.878 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:18:50.436+0000 7f72c8018dc0 0 load: jerasure load: lrc 2026-02-19T13:18:51.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_bad_size_shard_1 td/test-erasure-eio 2026-02-19T13:18:51.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:336: TEST_rados_get_bad_size_shard_1: local dir=td/test-erasure-eio 2026-02-19T13:18:51.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:337: TEST_rados_get_bad_size_shard_1: setup_osds 4 2026-02-19T13:18:51.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T13:18:51.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:18:51.879 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T13:18:51.880 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T13:18:51.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:18:51.882 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:51.883 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:51.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:51.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:18:51.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:18:51.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:18:51.884 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:18:51.884 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=180ebe6e-0274-4ec3-be72-b6b98c27bfcd 2026-02-19T13:18:51.885 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 180ebe6e-0274-4ec3-be72-b6b98c27bfcd 2026-02-19T13:18:51.885 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 180ebe6e-0274-4ec3-be72-b6b98c27bfcd' 2026-02-19T13:18:51.885 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:18:51.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC7DZdpXG9xNRAAFXLbf9QqMqlB71mUvRdN1g== 2026-02-19T13:18:51.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC7DZdpXG9xNRAAFXLbf9QqMqlB71mUvRdN1g=="}' 2026-02-19T13:18:51.895 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 180ebe6e-0274-4ec3-be72-b6b98c27bfcd -i td/test-erasure-eio/0/new.json 2026-02-19T13:18:52.115 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:18:52.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:18:52.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC7DZdpXG9xNRAAFXLbf9QqMqlB71mUvRdN1g== --osd-uuid 180ebe6e-0274-4ec3-be72-b6b98c27bfcd 2026-02-19T13:18:52.144 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:52.144+0000 7f276ef7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:52.145 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:52.146+0000 7f276ef7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:52.145 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:52.147+0000 7f276ef7f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:52.146 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:52.147+0000 7f276ef7f900 -1 bdev(0x564baffb0c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:18:52.147 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:52.147+0000 7f276ef7f900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:18:52.568 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:18:52.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:18:52.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:18:52.569 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:18:52.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:18:52.869 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:18:52.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:18:52.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:52.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:18:52.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:18:52.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:18:52.883 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:52.884+0000 7ff2f0e40900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:52.885 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:52.886+0000 7ff2f0e40900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:52.885 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:52.887+0000 7ff2f0e40900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:53.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:18:53.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:53.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:18:53.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:53.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:53.092 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:53.093 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:18:53.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:53.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:53.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:18:53.094 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:53.095+0000 7ff2f0e40900 -1 Falling back to public interface 2026-02-19T13:18:53.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:53.408 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:53.409+0000 7ff2f0e40900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:18:54.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:54.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:54.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:54.313 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:18:54.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:54.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:18:54.537 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1405114823,v1:127.0.0.1:6803/1405114823] [v2:127.0.0.1:6804/1405114823,v1:127.0.0.1:6805/1405114823] exists,up 180ebe6e-0274-4ec3-be72-b6b98c27bfcd 2026-02-19T13:18:54.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:54.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:54.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:54.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:18:54.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:18:54.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:54.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:54.539 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:18:54.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:18:54.541 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:18:54.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 2026-02-19T13:18:54.541 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b4097f13-a4a2-4ae7-ae9f-ea86f9db2999' 2026-02-19T13:18:54.541 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 2026-02-19T13:18:54.542 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:18:54.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQC+DZdp5AUKIRAA7oFl9QRSsvfC4BYANrnR6A== 2026-02-19T13:18:54.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQC+DZdp5AUKIRAA7oFl9QRSsvfC4BYANrnR6A=="}' 2026-02-19T13:18:54.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 -i td/test-erasure-eio/1/new.json 2026-02-19T13:18:54.776 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:18:54.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:18:54.788 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQC+DZdp5AUKIRAA7oFl9QRSsvfC4BYANrnR6A== --osd-uuid b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 2026-02-19T13:18:54.804 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:54.805+0000 7fabb2e4f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:54.805 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:54.807+0000 7fabb2e4f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:54.806 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:54.808+0000 7fabb2e4f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:54.806 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:54.808+0000 7fabb2e4f900 -1 bdev(0x561dd67eec00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:18:54.806 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:54.808+0000 7fabb2e4f900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:18:55.239 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:18:55.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:18:55.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:18:55.240 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:18:55.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:18:55.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:18:55.539 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:18:55.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:55.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:18:55.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:18:55.539 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:18:55.553 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:55.555+0000 7f079c6eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:55.555 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:55.556+0000 7f079c6eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:55.556 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:55.557+0000 7f079c6eb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:55.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:18:55.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:55.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:18:55.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:55.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:55.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:55.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:55.762 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:18:55.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:55.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:55.778 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:55.780+0000 7f079c6eb900 -1 Falling back to public interface 2026-02-19T13:18:55.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:56.179 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:56.180+0000 7f079c6eb900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:18:56.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:56.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:56.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:56.982 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:18:56.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:56.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:18:57.203 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2002386368,v1:127.0.0.1:6811/2002386368] [v2:127.0.0.1:6812/2002386368,v1:127.0.0.1:6813/2002386368] exists,up b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 2026-02-19T13:18:57.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:57.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:57.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:57.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:18:57.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:18:57.206 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:18:57.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ca8d2030-cc89-4a70-ab8c-8b89c4856715 2026-02-19T13:18:57.207 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 ca8d2030-cc89-4a70-ab8c-8b89c4856715 2026-02-19T13:18:57.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 ca8d2030-cc89-4a70-ab8c-8b89c4856715' 2026-02-19T13:18:57.207 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:18:57.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDBDZdpTOQWDRAAUyHDmSWntVJvHZ9Ix6Re/w== 2026-02-19T13:18:57.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDBDZdpTOQWDRAAUyHDmSWntVJvHZ9Ix6Re/w=="}' 2026-02-19T13:18:57.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ca8d2030-cc89-4a70-ab8c-8b89c4856715 -i td/test-erasure-eio/2/new.json 2026-02-19T13:18:57.439 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:18:57.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:18:57.452 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDBDZdpTOQWDRAAUyHDmSWntVJvHZ9Ix6Re/w== --osd-uuid ca8d2030-cc89-4a70-ab8c-8b89c4856715 2026-02-19T13:18:57.467 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:57.468+0000 7f829d0e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:57.468 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:57.470+0000 7f829d0e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:57.469 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:57.471+0000 7f829d0e4900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:57.469 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:57.471+0000 7f829d0e4900 -1 bdev(0x558b21ff0c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:18:57.470 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:57.471+0000 7f829d0e4900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:18:57.888 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:18:57.888 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:18:57.889 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:18:57.889 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:18:57.889 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:18:58.188 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:18:58.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:18:58.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:18:58.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:18:58.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:18:58.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:18:58.202 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:58.203+0000 7fe665887900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:58.204 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:58.205+0000 7fe665887900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:58.205 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:58.206+0000 7fe665887900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:18:58.378 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:58.379+0000 7fe665887900 -1 Falling back to public interface 2026-02-19T13:18:58.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:18:58.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:18:58.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:18:58.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:18:58.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:18:58.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:58.411 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:18:58.411 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:18:58.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:58.412 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:18:58.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:18:58.718 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:18:58.718+0000 7fe665887900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:18:59.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:18:59.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:18:59.636 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:18:59.636 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:18:59.636 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:18:59.637 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:18:59.857 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3366517077,v1:127.0.0.1:6819/3366517077] [v2:127.0.0.1:6820/3366517077,v1:127.0.0.1:6821/3366517077] exists,up ca8d2030-cc89-4a70-ab8c-8b89c4856715 2026-02-19T13:18:59.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:18:59.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:18:59.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:18:59.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:18:59.860 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:18:59.861 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:18:59.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:18:59.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:18:59.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:18:59.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:18:59.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:18:59.863 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a2385b92-e232-425b-bcb1-904185522226 2026-02-19T13:18:59.863 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 a2385b92-e232-425b-bcb1-904185522226 2026-02-19T13:18:59.864 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 a2385b92-e232-425b-bcb1-904185522226' 2026-02-19T13:18:59.864 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:18:59.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQDDDZdp2NULNBAAJ09WH46Aht9ykp2KHTahbA== 2026-02-19T13:18:59.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQDDDZdp2NULNBAAJ09WH46Aht9ykp2KHTahbA=="}' 2026-02-19T13:18:59.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a2385b92-e232-425b-bcb1-904185522226 -i td/test-erasure-eio/3/new.json 2026-02-19T13:19:00.094 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:19:00.106 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:19:00.107 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQDDDZdp2NULNBAAJ09WH46Aht9ykp2KHTahbA== --osd-uuid a2385b92-e232-425b-bcb1-904185522226 2026-02-19T13:19:00.122 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:00.122+0000 7f53a6061900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:00.123 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:00.124+0000 7f53a6061900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:00.124 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:00.125+0000 7f53a6061900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:00.125 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:00.125+0000 7f53a6061900 -1 bdev(0x5653c4122c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:19:00.125 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:00.125+0000 7f53a6061900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:19:00.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:19:00.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:19:00.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:19:00.551 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:19:00.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:19:00.852 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:19:00.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:19:00.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:00.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:19:00.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:19:00.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:19:00.866 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:00.867+0000 7f016b393900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:00.868 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:00.868+0000 7f016b393900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:00.869 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:00.869+0000 7f016b393900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:01.055 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:01.055+0000 7f016b393900 -1 Falling back to public interface 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:01.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:19:01.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:01.391 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:01.391+0000 7f016b393900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:19:02.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:02.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:02.292 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:19:02.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:02.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:02.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:19:02.511 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/1613560064,v1:127.0.0.1:6827/1613560064] [v2:127.0.0.1:6828/1613560064,v1:127.0.0.1:6829/1613560064] exists,up a2385b92-e232-425b-bcb1-904185522226 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:02.512 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:02.513 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:19:02.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:19:02.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:19:02.571 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:19:02.578 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:19:02.580 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:18:53.098+0000 7ff2f0e40900 0 load: jerasure load: lrc 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:339: TEST_rados_get_bad_size_shard_1: local poolname=pool-jerasure 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:340: TEST_rados_get_bad_size_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:19:02.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T13:19:02.859 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:19:02.859 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:19:03.219 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:19:03.230 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:19:04.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:19:04.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:04.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:04.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:04.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:04.232 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:04.232 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:04.233 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:04.233 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:04.233 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:04.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:04.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:04.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:04.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:04.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:04.312 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:04.532 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:04.533 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:04.533 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:04.533 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:04.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:04.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:04.533 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:04.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T13:19:04.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T13:19:04.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T13:19:04.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:04.610 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:04.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-19T13:19:04.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-19T13:19:04.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-19T13:19:04.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:04.690 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:04.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T13:19:04.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T13:19:04.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411331' 2026-02-19T13:19:04.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:04.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:04.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T13:19:04.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T13:19:04.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411331 3-111669149699' 2026-02-19T13:19:04.850 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:04.851 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T13:19:04.851 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:04.851 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:04.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-19T13:19:04.852 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:04.853 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-19T13:19:04.853 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803781 2026-02-19T13:19:04.854 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T13:19:04.854 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:05.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803781 2026-02-19T13:19:05.075 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:06.076 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:06.077 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:06.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-19T13:19:06.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:06.300 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-19T13:19:06.301 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:06.301 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:06.302 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574852 2026-02-19T13:19:06.302 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:06.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574852 2026-02-19T13:19:06.303 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-19T13:19:06.303 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574852 2026-02-19T13:19:06.304 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:06.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-19T13:19:06.525 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:06.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T13:19:06.525 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:06.526 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:06.527 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411331 2026-02-19T13:19:06.527 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:06.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411331 2026-02-19T13:19:06.529 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411331 2026-02-19T13:19:06.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T13:19:06.529 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:06.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T13:19:06.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:06.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T13:19:06.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:06.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:06.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149699 2026-02-19T13:19:06.751 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:06.752 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149699 2026-02-19T13:19:06.753 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149699 2026-02-19T13:19:06.753 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T13:19:06.753 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:06.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-19T13:19:06.977 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:06.977 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:06.977 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:07.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:07.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:07.275 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:07.275 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:07.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:07.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:07.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:07.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:07.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:07.493 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:07.494 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:07.494 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:07.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:07.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:07.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:07.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:342: TEST_rados_get_bad_size_shard_1: local shard_id=1 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:343: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 10 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=10 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-19T13:19:07.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-65583-1-10 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-65583-1-10 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-65583-1-10 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:19:07.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:07.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:19:07.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-65583-1-10 td/test-erasure-eio/ORIGINAL 2026-02-19T13:19:07.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-65583-1-10 td/test-erasure-eio 1 10 set 2026-02-19T13:19:07.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-1-10 2026-02-19T13:19:07.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:19:07.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:19:07.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:19:07.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T13:19:07.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:19:07.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-19T13:19:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:19:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T13:19:07.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:19:07.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-1-10 2026-02-19T13:19:07.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:19:07.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-1-10 2026-02-19T13:19:07.812 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-1-10 2026-02-19T13:19:07.812 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:19:08.038 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:19:08.039 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:08.039 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:19:08.039 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:19:08.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:19:08.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:19:08.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T13:19:08.039 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:19:08.324 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:19:08.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T13:19:08.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-19T13:19:08.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-19T13:19:08.338 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records in 2026-02-19T13:19:08.338 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records out 2026-02-19T13:19:08.338 INFO:tasks.workunit.client.0.trial111.stderr:10 bytes copied, 2.0067e-05 s, 498 kB/s 2026-02-19T13:19:08.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-65583-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-65583-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:08.339 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:08.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:08.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:08.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:08.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:08.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-65583-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:08.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:19:08.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:19:08.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:19:08.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:19:08.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:19:08.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-65583-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:08.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:19:08.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:08.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:19:08.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:19:08.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:19:08.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:19:08.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:08.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:08.813 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:19:08.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:08.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:19:08.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:19:08.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:19:08.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:19:08.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:19:08.825 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:08.826+0000 7fa15daeb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:08.827 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:08.827+0000 7fa15daeb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:08.828 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:08.828+0000 7fa15daeb900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:09.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:19:09.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:09.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:19:09.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:09.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:09.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:09.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:09.034 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:19:09.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:09.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:19:09.038 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:09.039+0000 7fa15daeb900 -1 Falling back to public interface 2026-02-19T13:19:09.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:09.379 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:09.379+0000 7fa15daeb900 -1 osd.1 33 log_to_monitors true 2026-02-19T13:19:10.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:10.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:10.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:10.254 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:10.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:10.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 39 up_thru 39 down_at 36 last_clean_interval [13,33) [v2:127.0.0.1:6810/1507926233,v1:127.0.0.1:6811/1507926233] [v2:127.0.0.1:6812/1507926233,v1:127.0.0.1:6813/1507926233] exists,up b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:10.478 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:10.479 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:10.479 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:10.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:10.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:10.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:10.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:10.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:10.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:10.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:10.556 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:10.556 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:10.775 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:10.775 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:10.776 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:10.776 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:10.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:10.776 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:10.776 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:10.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-19T13:19:10.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-19T13:19:10.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784' 2026-02-19T13:19:10.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:10.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:10.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724547 2026-02-19T13:19:10.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724547 2026-02-19T13:19:10.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-167503724547' 2026-02-19T13:19:10.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:10.937 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:11.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-19T13:19:11.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-19T13:19:11.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411335' 2026-02-19T13:19:11.014 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:11.014 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:11.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T13:19:11.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T13:19:11.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-167503724547 2-77309411335 3-111669149702' 2026-02-19T13:19:11.091 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:11.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-19T13:19:11.092 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:11.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:11.093 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803784 2026-02-19T13:19:11.093 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:11.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803784 2026-02-19T13:19:11.093 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-19T13:19:11.093 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803784 2026-02-19T13:19:11.094 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:11.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-19T13:19:11.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:11.316 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724547 2026-02-19T13:19:11.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:11.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:11.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-167503724547 2026-02-19T13:19:11.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:11.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724547 2026-02-19T13:19:11.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 167503724547' 2026-02-19T13:19:11.320 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 167503724547 2026-02-19T13:19:11.320 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:11.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724547 -lt 167503724547 2026-02-19T13:19:11.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:11.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411335 2026-02-19T13:19:11.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:11.547 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:11.548 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411335 2026-02-19T13:19:11.548 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:11.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411335 2026-02-19T13:19:11.549 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-19T13:19:11.550 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 77309411335 2026-02-19T13:19:11.550 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:11.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-19T13:19:11.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:11.769 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T13:19:11.769 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:11.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:11.770 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149702 2026-02-19T13:19:11.770 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:11.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149702 2026-02-19T13:19:11.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T13:19:11.771 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149702 2026-02-19T13:19:11.772 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:12.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-19T13:19:12.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:12.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:12.001 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:12.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:12.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:12.298 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:12.298 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:12.298 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:12.299 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:12.299 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:12.299 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:12.520 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:12.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:12.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:12.520 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:12.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:12.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:12.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:12.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:19:12.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:19:13.131 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:19:13.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-1-10 2026-02-19T13:19:13.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:19:13.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:19:13.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-1-10 2026-02-19T13:19:13.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:19:13.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:19:13.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-1-10 td/test-erasure-eio/COPY 2026-02-19T13:19:13.166 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:19:13.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:19:13.168 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-65583-1-10 td/test-erasure-eio 2 10 set 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-1-10 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:19:13.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=10 2026-02-19T13:19:13.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:19:13.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T13:19:13.172 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:19:13.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-1-10 2026-02-19T13:19:13.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:19:13.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-1-10 2026-02-19T13:19:13.172 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-1-10 2026-02-19T13:19:13.172 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:19:13.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:19:13.396 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:13.397 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:19:13.397 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:19:13.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:19:13.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:19:13.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-19T13:19:13.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:19:13.709 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:19:13.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T13:19:13.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 10 = 0 ']' 2026-02-19T13:19:13.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:224: set_size: dd if=/dev/urandom bs=10 count=1 of=td/test-erasure-eio/CORRUPT 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records in 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records out 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:10 bytes copied, 1.9506e-05 s, 513 kB/s 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-65583-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:19:13.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-65583-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:13.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:14.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:14.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-65583-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:14.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:19:14.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:19:14.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:19:14.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:19:14.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T13:19:14.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-65583-1-10 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:14.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T13:19:14.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:14.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:19:14.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:19:14.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:19:14.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:14.204 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:14.205 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr:start osd.2 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:19:14.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:19:14.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:19:14.220 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:14.220+0000 7f1cbac12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:14.221 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:14.221+0000 7f1cbac12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:14.222 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:14.222+0000 7f1cbac12900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:14.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:19:14.461 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:14.461+0000 7f1cbac12900 -1 Falling back to public interface 2026-02-19T13:19:14.646 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:14.839 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:14.839+0000 7f1cbac12900 -1 osd.2 45 log_to_monitors true 2026-02-19T13:19:15.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:15.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:15.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:15.648 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:15.648 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:15.649 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:19:15.871 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 up in weight 1 up_from 49 up_thru 36 down_at 46 last_clean_interval [18,45) [v2:127.0.0.1:6818/1597262643,v1:127.0.0.1:6819/1597262643] [v2:127.0.0.1:6820/1597262643,v1:127.0.0.1:6821/1597262643] exists,up ca8d2030-cc89-4a70-ab8c-8b89c4856715 2026-02-19T13:19:15.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:15.872 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:15.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:15.873 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:15.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:15.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:15.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:15.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:15.950 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:15.950 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:16.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:16.170 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:16.170 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:16.170 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:16.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:16.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:16.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:16.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-19T13:19:16.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-19T13:19:16.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787' 2026-02-19T13:19:16.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:16.250 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:16.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=167503724550 2026-02-19T13:19:16.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 167503724550 2026-02-19T13:19:16.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-167503724550' 2026-02-19T13:19:16.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:16.327 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:16.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397507 2026-02-19T13:19:16.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397507 2026-02-19T13:19:16.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-210453397507' 2026-02-19T13:19:16.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:16.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:16.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149705 2026-02-19T13:19:16.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149705 2026-02-19T13:19:16.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-167503724550 2-210453397507 3-111669149705' 2026-02-19T13:19:16.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:16.487 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-19T13:19:16.487 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:16.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:16.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803787 2026-02-19T13:19:16.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:16.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803787 2026-02-19T13:19:16.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-19T13:19:16.490 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803787 2026-02-19T13:19:16.490 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:16.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-19T13:19:16.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:17.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:17.712 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:17.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-19T13:19:17.933 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:17.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-167503724550 2026-02-19T13:19:17.934 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:17.935 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:17.935 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-167503724550 2026-02-19T13:19:17.935 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:17.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=167503724550 2026-02-19T13:19:17.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 167503724550' 2026-02-19T13:19:17.937 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 167503724550 2026-02-19T13:19:17.937 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:18.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 167503724550 -lt 167503724550 2026-02-19T13:19:18.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:18.156 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-210453397507 2026-02-19T13:19:18.156 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:18.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:18.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-210453397507 2026-02-19T13:19:18.157 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:18.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397507 2026-02-19T13:19:18.158 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 210453397507' 2026-02-19T13:19:18.158 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 210453397507 2026-02-19T13:19:18.159 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:18.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397507 -lt 210453397507 2026-02-19T13:19:18.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:18.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149705 2026-02-19T13:19:18.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:18.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:18.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149705 2026-02-19T13:19:18.384 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:18.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149705 2026-02-19T13:19:18.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-19T13:19:18.385 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149705 2026-02-19T13:19:18.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:18.604 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-19T13:19:18.605 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:18.605 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:18.605 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:18.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:18.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:18.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:18.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:18.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:18.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:18.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:18.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:19.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:19.119 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:19.119 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:19.119 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:19.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:19.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:19.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:19.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:19:19.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:19:19.735 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:19:19.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T13:19:19.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-1-10 2026-02-19T13:19:19.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:19:19.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:19:19.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-1-10 2026-02-19T13:19:19.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:19:19.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:19:19.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-1-10 td/test-erasure-eio/COPY 2026-02-19T13:19:19.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:19:19.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:19:19.771 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:19:19.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:344: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 0 2026-02-19T13:19:19.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T13:19:19.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T13:19:19.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-19T13:19:19.772 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=0 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=set 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-65583-1-0 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-65583-1-0 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-65583-1-0 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:19:19.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-65583-1-0 td/test-erasure-eio/ORIGINAL 2026-02-19T13:19:19.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-65583-1-0 td/test-erasure-eio 1 0 set 2026-02-19T13:19:19.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-1-0 2026-02-19T13:19:19.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:19:19.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:19:19.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:19:19.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-1-0 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-1-0 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-1-0 2026-02-19T13:19:19.793 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:19:20.016 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:19:20.017 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:20.017 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:19:20.017 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:19:20.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:19:20.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:19:20.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T13:19:20.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:19:20.301 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:19:20.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T13:19:20.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-19T13:19:20.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-19T13:19:20.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-65583-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:20.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:19:20.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:19:20.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:19:20.318 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-65583-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:20.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:20.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:20.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-65583-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:20.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:19:20.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:19:20.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:19:20.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:19:20.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:19:20.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-65583-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:20.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:19:20.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:20.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:19:20.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:19:20.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:19:20.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:20.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:20.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:20.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:19:20.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:19:20.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:19:20.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:19:20.800 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:19:20.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:20.800 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:19:20.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:19:20.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:19:20.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:19:20.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:19:20.811 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:20.811+0000 7f6f6c604900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:20.812 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:20.812+0000 7f6f6c604900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:20.813 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:20.813+0000 7f6f6c604900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:21.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:19:21.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:21.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:19:21.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:21.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:21.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:21.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:21.020 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:19:21.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:21.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:19:21.025 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:21.026+0000 7f6f6c604900 -1 Falling back to public interface 2026-02-19T13:19:21.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:21.384 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:21.385+0000 7f6f6c604900 -1 osd.1 54 log_to_monitors true 2026-02-19T13:19:22.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:22.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:22.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:22.241 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:22.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:22.241 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:19:22.462 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 59 up_thru 59 down_at 56 last_clean_interval [39,54) [v2:127.0.0.1:6810/3642857338,v1:127.0.0.1:6811/3642857338] [v2:127.0.0.1:6812/3642857338,v1:127.0.0.1:6813/3642857338] exists,up b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 2026-02-19T13:19:22.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:22.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:22.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:22.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:19:22.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:22.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:22.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:22.463 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:22.463 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:22.463 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:22.463 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:22.464 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:22.464 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:22.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:22.542 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:22.543 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:22.543 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:22.543 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:22.543 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:22.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:22.762 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:22.762 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:22.762 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:22.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:22.763 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:22.763 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:22.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803790 2026-02-19T13:19:22.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803790 2026-02-19T13:19:22.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803790' 2026-02-19T13:19:22.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:22.844 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:22.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070467 2026-02-19T13:19:22.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070467 2026-02-19T13:19:22.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803790 1-253403070467' 2026-02-19T13:19:22.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:22.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:23.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=210453397510 2026-02-19T13:19:23.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 210453397510 2026-02-19T13:19:23.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803790 1-253403070467 2-210453397510' 2026-02-19T13:19:23.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:23.002 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:23.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149709 2026-02-19T13:19:23.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149709 2026-02-19T13:19:23.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803790 1-253403070467 2-210453397510 3-111669149709' 2026-02-19T13:19:23.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:23.080 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803790 2026-02-19T13:19:23.080 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:23.081 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:23.081 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803790 2026-02-19T13:19:23.081 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:23.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803790 2026-02-19T13:19:23.082 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803790' 2026-02-19T13:19:23.083 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803790 2026-02-19T13:19:23.083 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:23.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803790 -lt 25769803790 2026-02-19T13:19:23.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:23.305 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-253403070467 2026-02-19T13:19:23.305 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:23.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:23.306 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-253403070467 2026-02-19T13:19:23.306 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:23.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070467 2026-02-19T13:19:23.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 253403070467' 2026-02-19T13:19:23.307 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 253403070467 2026-02-19T13:19:23.308 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:23.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070467 -lt 253403070467 2026-02-19T13:19:23.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:23.527 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-210453397510 2026-02-19T13:19:23.528 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:23.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:23.529 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-210453397510 2026-02-19T13:19:23.529 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:23.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=210453397510 2026-02-19T13:19:23.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 210453397510' 2026-02-19T13:19:23.530 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 210453397510 2026-02-19T13:19:23.531 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:23.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 210453397510 -lt 210453397510 2026-02-19T13:19:23.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:23.756 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149709 2026-02-19T13:19:23.756 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:23.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:23.757 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149709 2026-02-19T13:19:23.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:23.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149709 2026-02-19T13:19:23.760 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149709' 2026-02-19T13:19:23.760 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149709 2026-02-19T13:19:23.760 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:23.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149709 -lt 111669149709 2026-02-19T13:19:23.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:23.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:23.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:24.276 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:24.276 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:24.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:24.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:24.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:24.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:24.276 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:24.277 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:24.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:24.494 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:24.494 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:24.494 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:24.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:24.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:24.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:24.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:19:24.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:19:25.111 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:19:25.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-1-0 2026-02-19T13:19:25.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:19:25.124 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:19:25.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-1-0 2026-02-19T13:19:25.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:19:25.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:19:25.125 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-1-0 td/test-erasure-eio/COPY 2026-02-19T13:19:25.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:19:25.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:19:25.144 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-19T13:19:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-19T13:19:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-65583-1-0 td/test-erasure-eio 2 0 set 2026-02-19T13:19:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-1-0 2026-02-19T13:19:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:19:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=0 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=set 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-1-0 2026-02-19T13:19:25.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:19:25.148 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-1-0 2026-02-19T13:19:25.148 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-1-0 2026-02-19T13:19:25.148 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:19:25.373 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:19:25.373 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:25.373 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:19:25.373 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:19:25.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:19:25.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:19:25.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-19T13:19:25.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:19:25.685 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:19:25.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' set = add ']' 2026-02-19T13:19:25.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:220: set_size: '[' 0 = 0 ']' 2026-02-19T13:19:25.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:222: set_size: touch td/test-erasure-eio/CORRUPT 2026-02-19T13:19:25.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-65583-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:25.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:19:25.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:19:25.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:19:25.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:19:25.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-65583-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:25.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:19:25.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:19:25.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:19:25.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:19:25.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:19:25.703 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:25.703 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:25.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:25.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:25.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:26.009 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:26.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-65583-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:26.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:19:26.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:19:26.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:19:26.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:19:26.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T13:19:26.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-65583-1-0 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:26.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T13:19:26.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:26.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:19:26.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:19:26.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:26.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr:start osd.2 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:26.180 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:19:26.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:19:26.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:19:26.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:19:26.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:19:26.191 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:26.191+0000 7fad45e29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:26.193 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:26.193+0000 7fad45e29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:26.194 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:26.194+0000 7fad45e29900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:26.390 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:26.391+0000 7fad45e29900 -1 Falling back to public interface 2026-02-19T13:19:26.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:26.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:19:26.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:26.664 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:26.664+0000 7fad45e29900 -1 osd.2 63 log_to_monitors true 2026-02-19T13:19:27.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:27.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:27.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:27.633 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:27.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:27.633 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:19:27.855 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 up in weight 1 up_from 68 up_thru 56 down_at 65 last_clean_interval [49,63) [v2:127.0.0.1:6818/2433087715,v1:127.0.0.1:6819/2433087715] [v2:127.0.0.1:6820/2433087715,v1:127.0.0.1:6821/2433087715] exists,up ca8d2030-cc89-4a70-ab8c-8b89c4856715 2026-02-19T13:19:27.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:27.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:27.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:27.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:19:27.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:27.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:27.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:27.856 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:27.857 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:27.857 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:27.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:27.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:27.858 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:27.936 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:27.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:27.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:27.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:27.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:27.937 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:28.155 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:28.156 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:28.156 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:28.156 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:28.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:28.156 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:28.156 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:28.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803793 2026-02-19T13:19:28.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803793 2026-02-19T13:19:28.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803793' 2026-02-19T13:19:28.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:28.237 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:28.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=253403070470 2026-02-19T13:19:28.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 253403070470 2026-02-19T13:19:28.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803793 1-253403070470' 2026-02-19T13:19:28.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:28.314 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:28.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776131 2026-02-19T13:19:28.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776131 2026-02-19T13:19:28.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803793 1-253403070470 2-292057776131' 2026-02-19T13:19:28.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:28.392 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:28.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149712 2026-02-19T13:19:28.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149712 2026-02-19T13:19:28.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803793 1-253403070470 2-292057776131 3-111669149712' 2026-02-19T13:19:28.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:28.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803793 2026-02-19T13:19:28.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:28.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:28.473 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803793 2026-02-19T13:19:28.474 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:28.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803793 2026-02-19T13:19:28.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803793' 2026-02-19T13:19:28.475 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803793 2026-02-19T13:19:28.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:28.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803791 -lt 25769803793 2026-02-19T13:19:28.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:29.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:29.699 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:29.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803793 -lt 25769803793 2026-02-19T13:19:29.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:29.920 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-253403070470 2026-02-19T13:19:29.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:29.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:29.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-253403070470 2026-02-19T13:19:29.922 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:29.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=253403070470 2026-02-19T13:19:29.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 253403070470' 2026-02-19T13:19:29.923 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 253403070470 2026-02-19T13:19:29.924 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:30.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 253403070470 -lt 253403070470 2026-02-19T13:19:30.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:30.143 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776131 2026-02-19T13:19:30.143 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:30.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:30.145 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776131 2026-02-19T13:19:30.145 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:30.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776131 2026-02-19T13:19:30.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776131' 2026-02-19T13:19:30.146 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 292057776131 2026-02-19T13:19:30.147 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:30.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776131 -lt 292057776131 2026-02-19T13:19:30.368 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:30.368 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149712 2026-02-19T13:19:30.368 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:30.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:30.370 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149712 2026-02-19T13:19:30.370 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:30.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149712 2026-02-19T13:19:30.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149712' 2026-02-19T13:19:30.371 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149712 2026-02-19T13:19:30.371 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:30.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149712 -lt 111669149712 2026-02-19T13:19:30.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:30.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:30.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:30.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:30.887 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:30.887 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:30.887 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:30.887 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:30.887 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:30.888 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:30.888 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:31.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:31.104 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:31.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:31.105 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:31.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:31.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:31.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:31.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:19:31.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:19:31.712 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:19:31.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T13:19:31.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-1-0 2026-02-19T13:19:31.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:19:31.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:19:31.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-1-0 2026-02-19T13:19:31.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:19:31.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:19:31.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-1-0 td/test-erasure-eio/COPY 2026-02-19T13:19:31.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:19:31.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:19:31.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:19:31.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:345: TEST_rados_get_bad_size_shard_1: rados_get_data_bad_size td/test-erasure-eio 1 256 add 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:232: rados_get_data_bad_size: local dir=td/test-erasure-eio 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:233: rados_get_data_bad_size: shift 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:234: rados_get_data_bad_size: local shard_id=1 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:235: rados_get_data_bad_size: shift 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:236: rados_get_data_bad_size: local bytes=256 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:237: rados_get_data_bad_size: shift 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:238: rados_get_data_bad_size: local mode=add 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:240: rados_get_data_bad_size: local poolname=pool-jerasure 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:241: rados_get_data_bad_size: local objname=obj-size-65583-1-256 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:242: rados_get_data_bad_size: rados_put td/test-erasure-eio pool-jerasure obj-size-65583-1-256 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-size-65583-1-256 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:19:31.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-size-65583-1-256 td/test-erasure-eio/ORIGINAL 2026-02-19T13:19:31.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:246: rados_get_data_bad_size: set_size obj-size-65583-1-256 td/test-erasure-eio 1 256 add 2026-02-19T13:19:31.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-1-256 2026-02-19T13:19:31.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:19:31.765 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=1 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-1-256 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-1-256 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-1-256 2026-02-19T13:19:31.766 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:19:31.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:19:31.988 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:31.989 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:19:31.989 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:19:31.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:19:31.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:19:31.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=1 2026-02-19T13:19:31.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:19:32.275 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:19:32.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-19T13:19:32.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-65583-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:32.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:19:32.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:19:32.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:19:32.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:19:32.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-65583-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:32.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:32.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:32.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:32.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-65583-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:32.596 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:19:32.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:19:32.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:19:32.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:19:32.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:19:32.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-65583-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:32.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:32.701 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:19:32.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:19:32.703 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:19:32.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:32.703 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:19:32.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:19:32.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:19:32.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:19:32.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:19:32.714 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:32.715+0000 7f4429564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:32.715 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:32.716+0000 7f4429564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:32.716 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:32.717+0000 7f4429564900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:32.910 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:32.911+0000 7f4429564900 -1 Falling back to public interface 2026-02-19T13:19:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:19:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:19:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:32.927 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:32.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:32.928 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:19:32.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:32.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:19:33.148 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:33.249 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:33.251+0000 7f4429564900 -1 osd.1 73 log_to_monitors true 2026-02-19T13:19:34.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:34.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:34.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:34.149 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:34.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:34.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:19:34.369 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 77 up_thru 77 down_at 74 last_clean_interval [59,73) [v2:127.0.0.1:6810/1210029962,v1:127.0.0.1:6811/1210029962] [v2:127.0.0.1:6812/1210029962,v1:127.0.0.1:6813/1210029962] exists,up b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 2026-02-19T13:19:34.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:34.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:34.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:34.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:19:34.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:34.369 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:34.370 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:34.370 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:34.370 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:34.370 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:34.370 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:34.370 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:34.370 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:34.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:34.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:34.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:34.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:34.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:34.450 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:34.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:34.668 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:34.668 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:34.668 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:34.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:34.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:34.669 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:34.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803797 2026-02-19T13:19:34.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803797 2026-02-19T13:19:34.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797' 2026-02-19T13:19:34.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:34.747 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:34.824 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=330712481795 2026-02-19T13:19:34.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 330712481795 2026-02-19T13:19:34.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-330712481795' 2026-02-19T13:19:34.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:34.825 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:34.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776134 2026-02-19T13:19:34.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776134 2026-02-19T13:19:34.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-330712481795 2-292057776134' 2026-02-19T13:19:34.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:34.904 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:34.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149715 2026-02-19T13:19:34.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149715 2026-02-19T13:19:34.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803797 1-330712481795 2-292057776134 3-111669149715' 2026-02-19T13:19:34.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:34.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803797 2026-02-19T13:19:34.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:34.984 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:34.984 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803797 2026-02-19T13:19:34.984 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:34.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803797 2026-02-19T13:19:34.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803797' 2026-02-19T13:19:34.986 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803797 2026-02-19T13:19:34.986 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:35.207 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803794 -lt 25769803797 2026-02-19T13:19:35.208 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:36.208 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:36.209 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:36.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803797 -lt 25769803797 2026-02-19T13:19:36.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:36.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-330712481795 2026-02-19T13:19:36.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:36.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:36.433 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-330712481795 2026-02-19T13:19:36.433 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:36.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=330712481795 2026-02-19T13:19:36.434 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 330712481795' 2026-02-19T13:19:36.434 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 330712481795 2026-02-19T13:19:36.434 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:36.651 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 330712481795 -lt 330712481795 2026-02-19T13:19:36.652 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:36.652 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776134 2026-02-19T13:19:36.652 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:36.653 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:36.654 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776134 2026-02-19T13:19:36.654 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:36.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776134 2026-02-19T13:19:36.655 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776134' 2026-02-19T13:19:36.655 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 292057776134 2026-02-19T13:19:36.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:36.877 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776134 -lt 292057776134 2026-02-19T13:19:36.878 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:36.878 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149715 2026-02-19T13:19:36.878 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:36.879 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:36.879 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149715 2026-02-19T13:19:36.880 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:36.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149715 2026-02-19T13:19:36.880 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149715' 2026-02-19T13:19:36.880 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149715 2026-02-19T13:19:36.881 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:37.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149715 -lt 111669149715 2026-02-19T13:19:37.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:37.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:37.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:37.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:37.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:37.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:37.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:37.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:37.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:37.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:37.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:37.615 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:37.615 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:37.615 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:37.615 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:37.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:37.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:37.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:37.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-19T13:19:37.912 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records in 2026-02-19T13:19:37.912 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records out 2026-02-19T13:19:37.912 INFO:tasks.workunit.client.0.trial111.stderr:256 bytes copied, 1.8223e-05 s, 14.0 MB/s 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 1 obj-size-65583-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-size-65583-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:37.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:38.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:38.020 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-size-65583-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:38.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:19:38.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:19:38.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:19:38.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:19:38.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:19:38.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-size-65583-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:19:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:19:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:19:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:19:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:38.194 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:38.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:38.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:19:38.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:19:38.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:19:38.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:19:38.196 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:19:38.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:38.196 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:19:38.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:19:38.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:19:38.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:19:38.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:19:38.208 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:38.208+0000 7fb89aa16900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:38.210 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:38.210+0000 7fb89aa16900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:38.210 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:38.211+0000 7fb89aa16900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:38.386 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:38.385+0000 7fb89aa16900 -1 Falling back to public interface 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:38.416 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:19:38.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:38.791 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:38.791+0000 7fb89aa16900 -1 osd.1 78 log_to_monitors true 2026-02-19T13:19:39.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:39.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:39.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:39.640 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:39.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:39.640 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 82 up_thru 82 down_at 79 last_clean_interval [77,78) [v2:127.0.0.1:6810/3423816843,v1:127.0.0.1:6811/3423816843] [v2:127.0.0.1:6812/3423816843,v1:127.0.0.1:6813/3423816843] exists,up b4097f13-a4a2-4ae7-ae9f-ea86f9db2999 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:39.862 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:39.863 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:39.863 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:39.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:39.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:39.863 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:39.942 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:39.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:39.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:39.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:39.943 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:39.943 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:40.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:40.164 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:40.164 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:40.164 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:40.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:40.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:40.165 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:40.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803800 2026-02-19T13:19:40.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803800 2026-02-19T13:19:40.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800' 2026-02-19T13:19:40.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:40.244 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:40.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318275 2026-02-19T13:19:40.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318275 2026-02-19T13:19:40.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-352187318275' 2026-02-19T13:19:40.321 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:40.321 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:40.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=292057776137 2026-02-19T13:19:40.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 292057776137 2026-02-19T13:19:40.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-352187318275 2-292057776137' 2026-02-19T13:19:40.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:40.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:40.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149718 2026-02-19T13:19:40.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149718 2026-02-19T13:19:40.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803800 1-352187318275 2-292057776137 3-111669149718' 2026-02-19T13:19:40.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:40.476 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803800 2026-02-19T13:19:40.476 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:40.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:40.477 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803800 2026-02-19T13:19:40.477 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:40.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803800 2026-02-19T13:19:40.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803800' 2026-02-19T13:19:40.479 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803800 2026-02-19T13:19:40.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:40.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803797 -lt 25769803800 2026-02-19T13:19:40.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:41.700 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:41.701 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:41.924 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803800 -lt 25769803800 2026-02-19T13:19:41.925 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:41.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-352187318275 2026-02-19T13:19:41.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:41.926 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:41.927 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-352187318275 2026-02-19T13:19:41.927 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:41.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=352187318275 2026-02-19T13:19:41.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 352187318275' 2026-02-19T13:19:41.928 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 352187318275 2026-02-19T13:19:41.928 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:42.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318275 -lt 352187318275 2026-02-19T13:19:42.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:42.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-292057776137 2026-02-19T13:19:42.151 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:42.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:42.152 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-292057776137 2026-02-19T13:19:42.153 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:42.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=292057776137 2026-02-19T13:19:42.154 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 292057776137' 2026-02-19T13:19:42.154 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 292057776137 2026-02-19T13:19:42.154 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:42.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 292057776137 -lt 292057776137 2026-02-19T13:19:42.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:42.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149718 2026-02-19T13:19:42.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:42.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:42.377 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149718 2026-02-19T13:19:42.377 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:42.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149718 2026-02-19T13:19:42.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149718' 2026-02-19T13:19:42.378 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149718 2026-02-19T13:19:42.379 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:42.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149718 -lt 111669149718 2026-02-19T13:19:42.602 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:42.602 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:42.602 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:42.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:42.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:42.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:42.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:42.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:42.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:42.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:42.905 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:43.123 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:43.123 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:43.123 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:43.123 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:43.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:43.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:43.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:43.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:19:43.425 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:19:43.736 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:19:43.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:248: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-1-256 2026-02-19T13:19:43.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:19:43.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:19:43.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-1-256 2026-02-19T13:19:43.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:19:43.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:19:43.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-1-256 td/test-erasure-eio/COPY 2026-02-19T13:19:43.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:19:43.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:19:43.772 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: expr 1 + 1 2026-02-19T13:19:43.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:251: rados_get_data_bad_size: local another_shard_id=2 2026-02-19T13:19:43.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:252: rados_get_data_bad_size: set_size obj-size-65583-1-256 td/test-erasure-eio 2 256 add 2026-02-19T13:19:43.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:202: set_size: local objname=obj-size-65583-1-256 2026-02-19T13:19:43.773 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:203: set_size: shift 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:204: set_size: local dir=td/test-erasure-eio 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:205: set_size: shift 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:206: set_size: local shard_id=2 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:207: set_size: shift 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:208: set_size: local bytes=256 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:209: set_size: shift 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:210: set_size: local mode=add 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:212: set_size: local poolname=pool-jerasure 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: get_osds pool-jerasure obj-size-65583-1-256 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:19:43.774 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-size-65583-1-256 2026-02-19T13:19:43.775 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-size-65583-1-256 2026-02-19T13:19:43.775 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:19:43.996 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:19:43.996 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:43.997 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:19:43.997 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:19:43.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: initial_osds=('3' '1' '2') 2026-02-19T13:19:43.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:213: set_size: local -a initial_osds 2026-02-19T13:19:43.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:214: set_size: local osd_id=2 2026-02-19T13:19:43.997 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:215: set_size: ceph osd set noout 2026-02-19T13:19:44.309 INFO:tasks.workunit.client.0.trial111.stderr:noout is set 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:216: set_size: '[' add = add ']' 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:218: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-65583-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-65583-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:44.324 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:44.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:44.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-65583-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:44.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:19:44.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:19:44.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:19:44.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:19:44.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T13:19:44.632 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-65583-1-256 get-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:44.734 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:44.737 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:44.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:44.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:44.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:44.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:44.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:44.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr:start osd.2 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:44.738 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:19:44.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:19:44.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:19:44.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:19:44.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:19:44.751 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:44.751+0000 7f1e7f2f1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:44.752 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:44.753+0000 7f1e7f2f1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:44.753 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:44.754+0000 7f1e7f2f1900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:44.952 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:44.952+0000 7f1e7f2f1900 -1 Falling back to public interface 2026-02-19T13:19:44.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:19:44.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:44.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:19:44.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:44.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:44.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:44.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:44.958 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:19:44.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:44.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:19:45.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:45.320 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:45.321+0000 7f1e7f2f1900 -1 osd.2 85 log_to_monitors true 2026-02-19T13:19:46.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:46.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:46.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:46.178 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:46.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:46.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 up in weight 1 up_from 91 up_thru 79 down_at 88 last_clean_interval [68,85) [v2:127.0.0.1:6818/3647561626,v1:127.0.0.1:6819/3647561626] [v2:127.0.0.1:6820/3647561626,v1:127.0.0.1:6821/3647561626] exists,up ca8d2030-cc89-4a70-ab8c-8b89c4856715 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:46.405 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:46.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:46.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:46.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:46.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:46.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:46.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:46.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:46.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:46.487 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:46.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:46.713 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:46.713 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:46.713 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:46.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:46.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:46.713 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:46.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803803 2026-02-19T13:19:46.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803803 2026-02-19T13:19:46.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803803' 2026-02-19T13:19:46.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:46.795 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:46.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318278 2026-02-19T13:19:46.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318278 2026-02-19T13:19:46.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803803 1-352187318278' 2026-02-19T13:19:46.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:46.875 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:46.953 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=390842023939 2026-02-19T13:19:46.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 390842023939 2026-02-19T13:19:46.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803803 1-352187318278 2-390842023939' 2026-02-19T13:19:46.954 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:46.954 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:47.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149721 2026-02-19T13:19:47.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149721 2026-02-19T13:19:47.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803803 1-352187318278 2-390842023939 3-111669149721' 2026-02-19T13:19:47.033 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:47.033 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803803 2026-02-19T13:19:47.033 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:47.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:47.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803803 2026-02-19T13:19:47.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:47.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803803 2026-02-19T13:19:47.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803803' 2026-02-19T13:19:47.036 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803803 2026-02-19T13:19:47.036 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:47.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803803 -lt 25769803803 2026-02-19T13:19:47.257 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:47.257 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-352187318278 2026-02-19T13:19:47.257 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:47.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:47.259 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-352187318278 2026-02-19T13:19:47.259 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:47.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=352187318278 2026-02-19T13:19:47.260 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 352187318278' 2026-02-19T13:19:47.260 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 352187318278 2026-02-19T13:19:47.260 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:47.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318278 -lt 352187318278 2026-02-19T13:19:47.482 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:47.482 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-390842023939 2026-02-19T13:19:47.483 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:47.483 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:47.484 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-390842023939 2026-02-19T13:19:47.484 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:47.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=390842023939 2026-02-19T13:19:47.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 390842023939' 2026-02-19T13:19:47.485 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 390842023939 2026-02-19T13:19:47.486 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:47.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 390842023939 -lt 390842023939 2026-02-19T13:19:47.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:47.707 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149721 2026-02-19T13:19:47.707 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:47.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:47.708 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149721 2026-02-19T13:19:47.708 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:47.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149721 2026-02-19T13:19:47.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149721' 2026-02-19T13:19:47.710 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149721 2026-02-19T13:19:47.710 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:47.931 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149721 -lt 111669149721 2026-02-19T13:19:47.931 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:47.932 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:47.932 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:48.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:48.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:48.231 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:48.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:48.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:48.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:48.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:48.232 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:48.451 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:48.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:48.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:48.452 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:48.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:48.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:48.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:48.764 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:219: set_size: dd if=/dev/urandom bs=256 count=1 2026-02-19T13:19:48.766 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records in 2026-02-19T13:19:48.766 INFO:tasks.workunit.client.0.trial111.stderr:1+0 records out 2026-02-19T13:19:48.766 INFO:tasks.workunit.client.0.trial111.stderr:256 bytes copied, 1.9316e-05 s, 13.3 MB/s 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:226: set_size: objectstore_tool td/test-erasure-eio 2 obj-size-65583-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-size-65583-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:19:48.767 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:19:48.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:19:48.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:19:48.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:48.768 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:48.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:48.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:48.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:48.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:48.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-size-65583-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:48.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:19:48.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:19:48.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:19:48.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:19:48.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T13:19:48.871 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-size-65583-1-256 set-bytes td/test-erasure-eio/CORRUPT 2026-02-19T13:19:49.049 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T13:19:49.049 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:49.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:19:49.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:19:49.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:19:49.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:19:49.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:49.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:49.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:49.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:49.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:19:49.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:19:49.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:49.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:19:49.051 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:49.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:19:49.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:49.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:49.052 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:49.052 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:49.052 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:49.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:49.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:49.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:49.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:49.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:49.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:49.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:49.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:49.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:49.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:19:49.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:19:49.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:19:49.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:19:49.054 INFO:tasks.workunit.client.0.trial111.stderr:start osd.2 2026-02-19T13:19:49.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:49.055 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:19:49.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:19:49.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:19:49.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:19:49.055 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:19:49.059 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:49.059+0000 7fed23acd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:49.061 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:49.061+0000 7fed23acd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:49.062 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:49.062+0000 7fed23acd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:49.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:19:49.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:49.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:19:49.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:49.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:49.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:49.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:49.268 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:19:49.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:49.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:19:49.269 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:49.269+0000 7fed23acd900 -1 Falling back to public interface 2026-02-19T13:19:49.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:49.609 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:49.609+0000 7fed23acd900 -1 osd.2 92 log_to_monitors true 2026-02-19T13:19:50.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:19:50.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:50.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:19:50.489 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:50.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:50.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:19:50.715 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 up in weight 1 up_from 96 up_thru 79 down_at 93 last_clean_interval [91,92) [v2:127.0.0.1:6818/3907571209,v1:127.0.0.1:6819/3907571209] [v2:127.0.0.1:6820/3907571209,v1:127.0.0.1:6821/3907571209] exists,up ca8d2030-cc89-4a70-ab8c-8b89c4856715 2026-02-19T13:19:50.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:19:50.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:19:50.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:19:50.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:19:50.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:19:50.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:19:50.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:19:50.716 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:19:50.716 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:19:50.717 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:19:50.717 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:19:50.717 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:19:50.717 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:19:50.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:19:50.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:19:50.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:19:50.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:19:50.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:19:50.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:19:51.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:19:51.017 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:19:51.017 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:19:51.017 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:19:51.017 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:19:51.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:51.018 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:19:51.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803806 2026-02-19T13:19:51.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803806 2026-02-19T13:19:51.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803806' 2026-02-19T13:19:51.100 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:51.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:19:51.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=352187318281 2026-02-19T13:19:51.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 352187318281 2026-02-19T13:19:51.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803806 1-352187318281' 2026-02-19T13:19:51.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:51.178 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:19:51.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=412316860419 2026-02-19T13:19:51.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 412316860419 2026-02-19T13:19:51.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803806 1-352187318281 2-412316860419' 2026-02-19T13:19:51.258 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:19:51.258 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:19:51.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149724 2026-02-19T13:19:51.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149724 2026-02-19T13:19:51.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803806 1-352187318281 2-412316860419 3-111669149724' 2026-02-19T13:19:51.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:51.339 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803806 2026-02-19T13:19:51.339 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:51.339 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:19:51.340 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803806 2026-02-19T13:19:51.340 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:51.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803806 2026-02-19T13:19:51.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803806' 2026-02-19T13:19:51.342 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803806 2026-02-19T13:19:51.342 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:19:51.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803806 -lt 25769803806 2026-02-19T13:19:51.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:51.566 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-352187318281 2026-02-19T13:19:51.566 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:51.567 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:19:51.567 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-352187318281 2026-02-19T13:19:51.568 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:51.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=352187318281 2026-02-19T13:19:51.569 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 352187318281' 2026-02-19T13:19:51.569 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 352187318281 2026-02-19T13:19:51.569 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:51.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318279 -lt 352187318281 2026-02-19T13:19:51.787 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:52.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:19:52.789 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:53.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318279 -lt 352187318281 2026-02-19T13:19:53.010 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:19:54.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 299 -eq 0 ']' 2026-02-19T13:19:54.012 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:19:54.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 352187318281 -lt 352187318281 2026-02-19T13:19:54.232 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:54.233 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-412316860419 2026-02-19T13:19:54.233 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:54.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:19:54.235 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-412316860419 2026-02-19T13:19:54.235 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:54.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=412316860419 2026-02-19T13:19:54.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 412316860419' 2026-02-19T13:19:54.236 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 412316860419 2026-02-19T13:19:54.236 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:19:54.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 412316860419 -lt 412316860419 2026-02-19T13:19:54.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:19:54.458 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149724 2026-02-19T13:19:54.458 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:19:54.459 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:19:54.459 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149724 2026-02-19T13:19:54.459 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:19:54.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149724 2026-02-19T13:19:54.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149724' 2026-02-19T13:19:54.460 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149724 2026-02-19T13:19:54.461 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:19:54.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149725 -lt 111669149724 2026-02-19T13:19:54.681 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:19:54.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:54.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:54.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:19:54.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:19:54.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:19:54.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:19:54.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:19:54.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:19:54.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:19:54.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:19:55.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:19:55.197 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:19:55.197 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:19:55.198 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:19:55.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:19:55.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:19:55.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:19:55.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:227: set_size: rm -f td/test-erasure-eio/CORRUPT 2026-02-19T13:19:55.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:228: set_size: ceph osd unset noout 2026-02-19T13:19:55.808 INFO:tasks.workunit.client.0.trial111.stderr:noout is unset 2026-02-19T13:19:55.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:253: rados_get_data_bad_size: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T13:19:55.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:258: rados_get_data_bad_size: rados_get td/test-erasure-eio pool-jerasure obj-size-65583-1-256 2026-02-19T13:19:55.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:19:55.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:19:55.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-size-65583-1-256 2026-02-19T13:19:55.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:19:55.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:19:55.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-size-65583-1-256 td/test-erasure-eio/COPY 2026-02-19T13:19:55.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:19:55.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:19:55.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:264: rados_get_data_bad_size: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:19:55.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:346: TEST_rados_get_bad_size_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:19:55.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:19:55.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:19:56.067 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:19:56.079 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:19:56.346 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:19:56.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:19:56.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:19:56.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:19:56.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:19:56.359 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:56.359 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:56.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:56.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:56.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:56.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:56.466 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:19:56.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:19:56.467 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:19:56.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:19:56.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:19:56.468 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:19:56.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:19:56.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:19:56.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:19:56.470 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:19:56.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:19:56.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:19:56.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:19:56.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:19:56.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:56.488 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:56.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:19:56.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:19:56.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:19:56.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:19:56.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:19:56.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:19:56.490 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:19:56.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:19:56.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:19:56.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:19:56.491 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:19:56.491 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:19:56.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:19:56.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:19:56.491 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:19:56.492 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:19:56.492 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:19:56.492 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:19:56.493 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:19:56.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:19:56.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:19:56.494 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:19:56.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:19:56.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:19:56.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:19:56.496 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:19:56.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:19:56.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:19:56.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:19:56.499 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:19:56.499 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:56.499 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:56.499 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:19:56.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:19:56.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:19:56.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:19:56.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:19:56.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:56.501 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:56.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:19:56.502 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:19:56.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:19:56.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:19:56.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:19:56.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:56.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:56.523 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:56.523 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:56.523 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:56.523 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:56.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:19:56.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:19:56.547 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:19:56.548 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:19:56.548 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:19:56.548 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:19:56.548 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:19:56.549 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:19:56.549 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:19:56.549 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:19:56.549 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:19:56.549 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:56.550 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:56.550 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:19:56.550 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:19:56.550 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:19:56.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:19:56.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:19:56.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:19:56.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:19:56.607 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:19:56.608 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:19:56.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:19:56.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:19:56.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:19:56.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:19:56.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:19:56.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:19:56.667 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:19:56.777 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:19:56.777 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:56.777 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:56.778 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:56.778 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:56.778 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:56.778 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:56.778 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:19:56.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:19:56.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:19:56.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:19:56.902 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:19:56.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:19:57.915 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:19:57.915 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:19:57.915 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:19:57.916 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:19:57.916 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:57.917 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:57.917 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:19:57.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:19:57.917 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:19:57.972 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:19:57.979 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:19:57.980 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:19:56.540+0000 7f16f79eadc0 0 load: jerasure load: lrc 2026-02-19T13:19:57.981 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_0 td/test-erasure-eio 2026-02-19T13:19:57.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:275: TEST_rados_get_subread_eio_shard_0: local dir=td/test-erasure-eio 2026-02-19T13:19:57.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:276: TEST_rados_get_subread_eio_shard_0: setup_osds 4 2026-02-19T13:19:57.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T13:19:57.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:19:57.982 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T13:19:57.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T13:19:57.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:19:57.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:19:57.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:19:57.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:19:57.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:19:57.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:19:57.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:19:57.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:19:57.989 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:19:57.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:19:57.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:19:57.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:19:57.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:19:57.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:19:57.990 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:19:57.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=02a2715b-cf41-4495-b561-d552f0372e25 2026-02-19T13:19:57.991 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 02a2715b-cf41-4495-b561-d552f0372e25 2026-02-19T13:19:57.991 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 02a2715b-cf41-4495-b561-d552f0372e25' 2026-02-19T13:19:57.991 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:19:57.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQD9DZdp/G6LOxAAqnzAawphbQC+L2wsg93ECQ== 2026-02-19T13:19:57.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQD9DZdp/G6LOxAAqnzAawphbQC+L2wsg93ECQ=="}' 2026-02-19T13:19:57.999 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 02a2715b-cf41-4495-b561-d552f0372e25 -i td/test-erasure-eio/0/new.json 2026-02-19T13:19:58.214 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:19:58.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:19:58.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQD9DZdp/G6LOxAAqnzAawphbQC+L2wsg93ECQ== --osd-uuid 02a2715b-cf41-4495-b561-d552f0372e25 2026-02-19T13:19:58.241 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:58.242+0000 7f79e10d0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:58.243 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:58.243+0000 7f79e10d0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:58.244 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:58.244+0000 7f79e10d0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:58.244 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:58.244+0000 7f79e10d0900 -1 bdev(0x55ce5d97ec00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:19:58.244 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:58.244+0000 7f79e10d0900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:19:58.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:19:58.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:19:58.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:19:58.657 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:19:58.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:19:58.958 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:19:58.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:19:58.959 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:19:58.960 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:19:58.960 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:19:58.960 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:19:58.974 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:58.974+0000 7f1080b37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:58.975 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:58.975+0000 7f1080b37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:58.976 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:58.976+0000 7f1080b37900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:19:59.170 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:59.171+0000 7f1080b37900 -1 Falling back to public interface 2026-02-19T13:19:59.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:19:59.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:19:59.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:19:59.477 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:19:59.477+0000 7f1080b37900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:20:00.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:00.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:00.403 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:00.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:00.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:00.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:20:00.623 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3135250689,v1:127.0.0.1:6803/3135250689] [v2:127.0.0.1:6804/3135250689,v1:127.0.0.1:6805/3135250689] exists,up 02a2715b-cf41-4495-b561-d552f0372e25 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:00.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:00.625 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:20:00.626 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:00.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=1185cdfd-319a-4702-a4d0-8f6f4fb40c01 2026-02-19T13:20:00.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 1185cdfd-319a-4702-a4d0-8f6f4fb40c01' 2026-02-19T13:20:00.628 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 1185cdfd-319a-4702-a4d0-8f6f4fb40c01 2026-02-19T13:20:00.628 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:00.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAADpdpjU8eJhAADGsypFtlcCFvbyg+8wRyrw== 2026-02-19T13:20:00.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAADpdpjU8eJhAADGsypFtlcCFvbyg+8wRyrw=="}' 2026-02-19T13:20:00.639 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 1185cdfd-319a-4702-a4d0-8f6f4fb40c01 -i td/test-erasure-eio/1/new.json 2026-02-19T13:20:00.858 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:00.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:20:00.870 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAADpdpjU8eJhAADGsypFtlcCFvbyg+8wRyrw== --osd-uuid 1185cdfd-319a-4702-a4d0-8f6f4fb40c01 2026-02-19T13:20:00.885 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:00.885+0000 7fa94f251900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:00.886 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:00.887+0000 7fa94f251900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:00.887 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:00.888+0000 7fa94f251900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:00.888 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:00.888+0000 7fa94f251900 -1 bdev(0x562850f3ac00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:00.888 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:00.888+0000 7fa94f251900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:20:01.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:20:01.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:01.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:20:01.300 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:20:01.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:01.600 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:20:01.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:20:01.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:01.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:01.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:01.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:01.614 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:01.615+0000 7f2edb5c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:01.616 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:01.616+0000 7f2edb5c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:01.617 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:01.617+0000 7f2edb5c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:01.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:20:01.822 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:01.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:20:01.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:01.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:01.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:01.823 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:01.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:01.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:01.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:20:01.826 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:01.827+0000 7f2edb5c6900 -1 Falling back to public interface 2026-02-19T13:20:02.042 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:02.189 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:02.189+0000 7f2edb5c6900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:20:03.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:03.044 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:03.045 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:03.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:03.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:03.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:20:03.266 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 12 up_thru 12 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/3253029009,v1:127.0.0.1:6811/3253029009] [v2:127.0.0.1:6812/3253029009,v1:127.0.0.1:6813/3253029009] exists,up 1185cdfd-319a-4702-a4d0-8f6f4fb40c01 2026-02-19T13:20:03.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:03.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:03.268 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:03.269 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:03.269 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:03.269 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:03.269 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=ec1537d1-b7a9-4e00-8c11-828d4ca43fa1 2026-02-19T13:20:03.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 ec1537d1-b7a9-4e00-8c11-828d4ca43fa1' 2026-02-19T13:20:03.271 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 ec1537d1-b7a9-4e00-8c11-828d4ca43fa1 2026-02-19T13:20:03.271 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:03.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQADDpdpa+HeEBAA/LP+nQQM7PVCRcs4JT8KhQ== 2026-02-19T13:20:03.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQADDpdpa+HeEBAA/LP+nQQM7PVCRcs4JT8KhQ=="}' 2026-02-19T13:20:03.282 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new ec1537d1-b7a9-4e00-8c11-828d4ca43fa1 -i td/test-erasure-eio/2/new.json 2026-02-19T13:20:03.519 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:20:03.530 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:20:03.531 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQADDpdpa+HeEBAA/LP+nQQM7PVCRcs4JT8KhQ== --osd-uuid ec1537d1-b7a9-4e00-8c11-828d4ca43fa1 2026-02-19T13:20:03.545 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:03.546+0000 7f90b50ab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:03.547 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:03.547+0000 7f90b50ab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:03.548 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:03.548+0000 7f90b50ab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:03.548 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:03.548+0000 7f90b50ab900 -1 bdev(0x55b26ff3ec00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:03.548 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:03.549+0000 7f90b50ab900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:20:03.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:20:03.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:03.968 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:20:03.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:20:03.969 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:04.274 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:20:04.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:20:04.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:04.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:04.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:04.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:04.289 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:04.290+0000 7f6b9a883900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:04.290 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:04.291+0000 7f6b9a883900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:04.291 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:04.292+0000 7f6b9a883900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:04.485 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:04.486+0000 7f6b9a883900 -1 Falling back to public interface 2026-02-19T13:20:04.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:20:04.493 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:04.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:20:04.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:04.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:04.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:04.494 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:04.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:04.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:04.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:20:04.713 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:04.889 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:04.890+0000 7f6b9a883900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:20:05.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:05.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:05.715 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:05.716 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:05.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:05.716 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:20:05.937 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4233964815,v1:127.0.0.1:6819/4233964815] [v2:127.0.0.1:6820/4233964815,v1:127.0.0.1:6821/4233964815] exists,up ec1537d1-b7a9-4e00-8c11-828d4ca43fa1 2026-02-19T13:20:05.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:05.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:05.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:05.937 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:05.938 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:05.939 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:20:05.940 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:05.940 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=44074ed0-cc2d-4d71-a966-5bfbedd09818 2026-02-19T13:20:05.940 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 44074ed0-cc2d-4d71-a966-5bfbedd09818 2026-02-19T13:20:05.941 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 44074ed0-cc2d-4d71-a966-5bfbedd09818' 2026-02-19T13:20:05.941 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:05.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAFDpdpWAfXOBAAuiHcxo2IZj+k07kf29AKVA== 2026-02-19T13:20:05.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAFDpdpWAfXOBAAuiHcxo2IZj+k07kf29AKVA=="}' 2026-02-19T13:20:05.952 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 44074ed0-cc2d-4d71-a966-5bfbedd09818 -i td/test-erasure-eio/3/new.json 2026-02-19T13:20:06.174 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:20:06.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:20:06.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAFDpdpWAfXOBAAuiHcxo2IZj+k07kf29AKVA== --osd-uuid 44074ed0-cc2d-4d71-a966-5bfbedd09818 2026-02-19T13:20:06.202 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:06.203+0000 7f015b561900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:06.203 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:06.204+0000 7f015b561900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:06.204 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:06.205+0000 7f015b561900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:06.204 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:06.206+0000 7f015b561900 -1 bdev(0x56437fc1ac00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:06.204 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:06.206+0000 7f015b561900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:20:06.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:20:06.608 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:06.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:20:06.609 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:20:06.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:06.907 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:20:06.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:20:06.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:06.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:06.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:06.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:06.922 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:06.923+0000 7fa33de1f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:06.924 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:06.925+0000 7fa33de1f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:06.925 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:06.926+0000 7fa33de1f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:07.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:20:07.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:07.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:20:07.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:07.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:07.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:07.132 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:07.132 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:07.133 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:07.133 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:20:07.133 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:07.133+0000 7fa33de1f900 -1 Falling back to public interface 2026-02-19T13:20:07.360 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:07.490 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:07.491+0000 7fa33de1f900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:20:08.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:08.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:08.362 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:08.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:08.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:08.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:20:08.588 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/320601213,v1:127.0.0.1:6827/320601213] [v2:127.0.0.1:6828/320601213,v1:127.0.0.1:6829/320601213] exists,up 44074ed0-cc2d-4d71-a966-5bfbedd09818 2026-02-19T13:20:08.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:08.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:08.590 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:08.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:20:08.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:20:08.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:20:08.590 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:08.590 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:08.591 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:08.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:20:08.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:20:08.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:20:08.649 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:20:08.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:20:08.658 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:19:59.173+0000 7f1080b37900 0 load: jerasure load: lrc 2026-02-19T13:20:08.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:278: TEST_rados_get_subread_eio_shard_0: local poolname=pool-jerasure 2026-02-19T13:20:08.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:279: TEST_rados_get_subread_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T13:20:08.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:20:08.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:20:08.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T13:20:08.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:20:08.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T13:20:08.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:20:08.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T13:20:08.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:20:08.912 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:20:09.277 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:20:09.288 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:20:10.289 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:20:10.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:20:10.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:20:10.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:20:10.290 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:20:10.291 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:20:10.291 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:20:10.291 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:20:10.291 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:20:10.291 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:20:10.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:20:10.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:20:10.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:20:10.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:20:10.373 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:10.373 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:10.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:10.597 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:10.597 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:20:10.597 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:20:10.597 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:10.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:10.598 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:10.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-19T13:20:10.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-19T13:20:10.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780' 2026-02-19T13:20:10.678 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:10.678 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:10.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=51539607556 2026-02-19T13:20:10.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 51539607556 2026-02-19T13:20:10.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607556' 2026-02-19T13:20:10.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:10.756 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:10.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T13:20:10.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T13:20:10.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331' 2026-02-19T13:20:10.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:10.834 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:20:10.914 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149700 2026-02-19T13:20:10.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149700 2026-02-19T13:20:10.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-51539607556 2-77309411331 3-111669149700' 2026-02-19T13:20:10.915 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:10.915 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-19T13:20:10.915 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:10.916 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:10.916 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803780 2026-02-19T13:20:10.917 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:10.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803780 2026-02-19T13:20:10.918 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-19T13:20:10.918 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803780 2026-02-19T13:20:10.918 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:11.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-19T13:20:11.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:12.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:12.140 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:12.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-19T13:20:12.363 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:12.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-51539607556 2026-02-19T13:20:12.364 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:12.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:12.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-51539607556 2026-02-19T13:20:12.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:12.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=51539607556 2026-02-19T13:20:12.367 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 51539607556 2026-02-19T13:20:12.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 51539607556' 2026-02-19T13:20:12.368 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:12.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 51539607556 -lt 51539607556 2026-02-19T13:20:12.587 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:12.588 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T13:20:12.588 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:12.589 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:12.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411331 2026-02-19T13:20:12.590 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:12.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411331 2026-02-19T13:20:12.591 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411331 2026-02-19T13:20:12.591 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T13:20:12.591 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:12.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T13:20:12.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:12.811 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149700 2026-02-19T13:20:12.811 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:12.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:20:12.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149700 2026-02-19T13:20:12.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:12.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149700 2026-02-19T13:20:12.814 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149700 2026-02-19T13:20:12.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149700' 2026-02-19T13:20:12.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:20:13.034 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149700 -lt 111669149700 2026-02-19T13:20:13.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:20:13.035 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:20:13.036 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:20:13.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:20:13.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:20:13.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:20:13.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:20:13.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:20:13.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:20:13.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:20:13.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:20:13.550 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:20:13.551 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:20:13.551 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:20:13.551 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:20:13.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:20:13.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:20:13.855 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:281: TEST_rados_get_subread_eio_shard_0: local shard_id=0 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:282: TEST_rados_get_subread_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T13:20:13.856 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-65583-0 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-65583-0 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-65583-0 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:13.857 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:20:13.858 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-65583-0 td/test-erasure-eio/ORIGINAL 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-65583-0 td/test-erasure-eio 0 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-65583-0 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:20:13.873 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:20:13.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:20:13.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:20:13.874 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:20:13.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-65583-0 2026-02-19T13:20:13.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:20:13.874 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-0 2026-02-19T13:20:13.874 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-0 2026-02-19T13:20:13.874 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:20:14.097 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T13:20:14.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:20:14.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T13:20:14.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:20:14.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T13:20:14.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:20:14.098 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:20:14.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:20:14.099 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T13:20:14.099 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:20:14.099 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:20:14.100 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:14.100 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:14.100 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:14.100 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:20:14.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:20:14.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:20:14.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:20:14.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-65583-0 0 2026-02-19T13:20:14.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-65583-0 2026-02-19T13:20:14.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:20:14.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:20:14.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-65583-0 2026-02-19T13:20:14.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:20:14.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:20:14.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-65583-0 td/test-erasure-eio/COPY 2026-02-19T13:20:14.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:20:14.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:20:14.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-19T13:20:14.253 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 0 + 1 2026-02-19T13:20:14.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=1 2026-02-19T13:20:14.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-65583-0 td/test-erasure-eio 1 2026-02-19T13:20:14.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:20:14.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:20:14.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:20:14.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:20:14.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-65583-0 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-65583-0 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-0 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-0 2026-02-19T13:20:14.256 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:20:14.476 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:20:14.477 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T13:20:14.479 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:20:14.480 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:20:14.481 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:14.481 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:14.481 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:14.481 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:20:14.481 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:20:14.543 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:20:14.544 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:20:14.545 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:20:14.545 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:20:14.545 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:20:14.545 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:20:14.545 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:14.545 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:14.545 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:14.546 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:20:14.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:20:14.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-65583-0 1 2026-02-19T13:20:14.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 0 -eq 1 -a 1 -eq 2 ']' 2026-02-19T13:20:14.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:192: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-65583-0 fail 2026-02-19T13:20:14.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:20:14.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:20:14.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-65583-0 2026-02-19T13:20:14.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=fail 2026-02-19T13:20:14.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' fail = fail ']' 2026-02-19T13:20:14.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:114: rados_get: rados --pool pool-jerasure get obj-eio-65583-0 td/test-erasure-eio/COPY 2026-02-19T13:20:14.624 INFO:tasks.workunit.client.0.trial111.stderr:error getting pool-jerasure/obj-eio-65583-0: (5) Input/output error 2026-02-19T13:20:14.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:115: rados_get: return 2026-02-19T13:20:14.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:20:14.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:283: TEST_rados_get_subread_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:20:14.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:20:14.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:20:14.893 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:20:14.904 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:20:15.171 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:20:15.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:20:15.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:20:15.182 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:20:15.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:20:15.183 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:20:15.183 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:20:15.183 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:20:15.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:20:15.184 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:20:15.290 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:20:15.290 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:20:15.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:20:15.291 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:20:15.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:20:15.292 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:20:15.293 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:20:15.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:20:15.293 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:20:15.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:20:15.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:20:15.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:20:15.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:20:15.295 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:20:15.303 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:20:15.304 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:15.304 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:15.304 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:20:15.305 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:20:15.306 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:20:15.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:20:15.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:20:15.307 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:20:15.307 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:20:15.308 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:20:15.309 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:20:15.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:20:15.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:20:15.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:20:15.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:20:15.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:20:15.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:20:15.311 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:20:15.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:20:15.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:20:15.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:20:15.315 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:20:15.315 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:15.315 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:15.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:20:15.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:20:15.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:20:15.316 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:20:15.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:20:15.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:15.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:15.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:20:15.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:20:15.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:20:15.319 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:20:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:20:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:20:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:20:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:20:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:20:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:20:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:20:15.320 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:20:15.338 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:20:15.339 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:15.339 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:15.339 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:15.339 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:15.339 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:15.339 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:15.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:20:15.362 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:20:15.363 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:20:15.363 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:20:15.363 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:20:15.363 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:20:15.363 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:20:15.364 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:20:15.365 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:20:15.365 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:20:15.365 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:15.365 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:15.365 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:15.365 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:20:15.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:20:15.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:20:15.424 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:20:15.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:20:15.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:20:15.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:20:15.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:20:15.425 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:20:15.425 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:20:15.425 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:20:15.426 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:15.426 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:15.427 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:15.427 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:20:15.427 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:20:15.427 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:20:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:20:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:20:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:20:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:20:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:20:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:20:15.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:20:15.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:20:15.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:15.597 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:15.597 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:15.598 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:15.598 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:15.598 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:15.599 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:20:15.599 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:20:15.615 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:20:15.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:20:15.721 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:20:15.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:20:16.733 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:20:16.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:20:16.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:20:16.735 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:16.735 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:16.735 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:16.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:20:16.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:20:16.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:20:16.790 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:20:16.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:20:16.800 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:20:15.355+0000 7f2af7f40dc0 0 load: jerasure load: lrc 2026-02-19T13:20:16.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_eio_shard_1 td/test-erasure-eio 2026-02-19T13:20:16.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:287: TEST_rados_get_subread_eio_shard_1: local dir=td/test-erasure-eio 2026-02-19T13:20:16.802 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:288: TEST_rados_get_subread_eio_shard_1: setup_osds 4 2026-02-19T13:20:16.802 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T13:20:16.802 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:20:16.802 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T13:20:16.802 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T13:20:16.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:16.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:20:16.806 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:20:16.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:16.808 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=733fa6fd-9b21-4274-9361-d0f84ed2170d 2026-02-19T13:20:16.809 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 733fa6fd-9b21-4274-9361-d0f84ed2170d' 2026-02-19T13:20:16.810 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 733fa6fd-9b21-4274-9361-d0f84ed2170d 2026-02-19T13:20:16.810 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:16.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAQDpdpWRDEMBAAgSLF6JXMnYzSEOIDyGRXdA== 2026-02-19T13:20:16.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAQDpdpWRDEMBAAgSLF6JXMnYzSEOIDyGRXdA=="}' 2026-02-19T13:20:16.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 733fa6fd-9b21-4274-9361-d0f84ed2170d -i td/test-erasure-eio/0/new.json 2026-02-19T13:20:17.039 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:17.049 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:20:17.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAQDpdpWRDEMBAAgSLF6JXMnYzSEOIDyGRXdA== --osd-uuid 733fa6fd-9b21-4274-9361-d0f84ed2170d 2026-02-19T13:20:17.065 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.065+0000 7f77f2eb0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:17.066 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.067+0000 7f77f2eb0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:17.067 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.068+0000 7f77f2eb0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:17.067 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.068+0000 7f77f2eb0900 -1 bdev(0x55803bf72c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:17.068 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.068+0000 7f77f2eb0900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:20:17.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:20:17.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:17.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:20:17.461 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:20:17.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:17.761 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:20:17.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:20:17.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:17.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:17.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:17.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:17.775 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.776+0000 7fc61ef91900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:17.776 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.777+0000 7fc61ef91900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:17.777 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.778+0000 7fc61ef91900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:17.943 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:17.943+0000 7fc61ef91900 -1 Falling back to public interface 2026-02-19T13:20:17.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:20:17.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:17.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:20:17.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:17.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:17.987 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:17.987 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:17.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:17.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:17.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:20:18.210 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:18.281 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:18.282+0000 7fc61ef91900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:20:19.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:19.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:19.212 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:19.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:19.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:19.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:20:19.435 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/1137346984,v1:127.0.0.1:6803/1137346984] [v2:127.0.0.1:6804/1137346984,v1:127.0.0.1:6805/1137346984] exists,up 733fa6fd-9b21-4274-9361-d0f84ed2170d 2026-02-19T13:20:19.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:19.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:19.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:19.440 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:19.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:19.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:20:19.443 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:19.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=39cdc573-61e1-4176-88dd-c4a01c0a89ce 2026-02-19T13:20:19.443 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 39cdc573-61e1-4176-88dd-c4a01c0a89ce 2026-02-19T13:20:19.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 39cdc573-61e1-4176-88dd-c4a01c0a89ce' 2026-02-19T13:20:19.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:19.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQATDpdpA9LeGhAACsRhRunov5NBF9ay46tlcg== 2026-02-19T13:20:19.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQATDpdpA9LeGhAACsRhRunov5NBF9ay46tlcg=="}' 2026-02-19T13:20:19.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 39cdc573-61e1-4176-88dd-c4a01c0a89ce -i td/test-erasure-eio/1/new.json 2026-02-19T13:20:19.670 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:19.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:20:19.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQATDpdpA9LeGhAACsRhRunov5NBF9ay46tlcg== --osd-uuid 39cdc573-61e1-4176-88dd-c4a01c0a89ce 2026-02-19T13:20:19.698 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:19.698+0000 7f061666a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:19.699 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:19.700+0000 7f061666a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:19.700 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:19.701+0000 7f061666a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:19.701 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:19.701+0000 7f061666a900 -1 bdev(0x560871432c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:19.701 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:19.701+0000 7f061666a900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:20:20.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:20:20.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:20.168 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:20:20.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:20:20.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:20.466 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:20:20.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:20:20.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:20.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:20.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:20.466 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:20.480 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:20.481+0000 7f9e47937900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:20.482 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:20.482+0000 7f9e47937900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:20.483 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:20.483+0000 7f9e47937900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:20.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:20:20.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:20.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:20:20.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:20.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:20.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:20.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:20.690 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:20.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:20.690 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:20:20.717 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:20.718+0000 7f9e47937900 -1 Falling back to public interface 2026-02-19T13:20:20.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:21.098 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:21.098+0000 7f9e47937900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:20:21.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:21.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:21.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:21.910 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:21.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:21.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:20:22.131 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/1058856439,v1:127.0.0.1:6811/1058856439] [v2:127.0.0.1:6812/1058856439,v1:127.0.0.1:6813/1058856439] exists,up 39cdc573-61e1-4176-88dd-c4a01c0a89ce 2026-02-19T13:20:22.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:22.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:22.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:22.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:22.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:20:22.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:22.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:22.136 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:22.137 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:22.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:22.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:22.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:22.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:20:22.138 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:22.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=edd5dbe9-9d09-48d7-9fa4-2353dacb3c1d 2026-02-19T13:20:22.138 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 edd5dbe9-9d09-48d7-9fa4-2353dacb3c1d 2026-02-19T13:20:22.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 edd5dbe9-9d09-48d7-9fa4-2353dacb3c1d' 2026-02-19T13:20:22.139 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:22.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAWDpdp0svECBAAe392N/uPT4CwV0OAggnnaQ== 2026-02-19T13:20:22.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAWDpdp0svECBAAe392N/uPT4CwV0OAggnnaQ=="}' 2026-02-19T13:20:22.147 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new edd5dbe9-9d09-48d7-9fa4-2353dacb3c1d -i td/test-erasure-eio/2/new.json 2026-02-19T13:20:22.373 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:20:22.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:20:22.385 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAWDpdp0svECBAAe392N/uPT4CwV0OAggnnaQ== --osd-uuid edd5dbe9-9d09-48d7-9fa4-2353dacb3c1d 2026-02-19T13:20:22.400 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:22.400+0000 7f6d51195900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:22.401 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:22.401+0000 7f6d51195900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:22.402 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:22.402+0000 7f6d51195900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:22.402 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:22.402+0000 7f6d51195900 -1 bdev(0x55af19996c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:22.402 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:22.403+0000 7f6d51195900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:20:22.833 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:20:22.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:22.834 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:20:22.835 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:20:22.835 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:23.137 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:20:23.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:20:23.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:23.138 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:23.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:23.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:23.152 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:23.152+0000 7f8b3691d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:23.153 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:23.154+0000 7f8b3691d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:23.154 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:23.154+0000 7f8b3691d900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:23.310 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:23.310+0000 7f8b3691d900 -1 Falling back to public interface 2026-02-19T13:20:23.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:20:23.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:23.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:20:23.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:23.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:23.358 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:23.358 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:23.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:23.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:23.359 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:20:23.580 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:23.662 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:23.662+0000 7f8b3691d900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:20:24.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:24.582 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:24.583 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:24.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:24.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:24.583 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:20:24.809 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/4164127339,v1:127.0.0.1:6819/4164127339] [v2:127.0.0.1:6820/4164127339,v1:127.0.0.1:6821/4164127339] exists,up edd5dbe9-9d09-48d7-9fa4-2353dacb3c1d 2026-02-19T13:20:24.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:24.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:24.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:24.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:24.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:20:24.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:24.813 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:24.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:24.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:24.816 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9cd9be71-cc4f-4918-bf1f-e12be610e793 2026-02-19T13:20:24.817 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 9cd9be71-cc4f-4918-bf1f-e12be610e793 2026-02-19T13:20:24.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 9cd9be71-cc4f-4918-bf1f-e12be610e793' 2026-02-19T13:20:24.817 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:24.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAYDpdpIPofMRAA5SS4fMnpADK/Vy38I7Rm4Q== 2026-02-19T13:20:24.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAYDpdpIPofMRAA5SS4fMnpADK/Vy38I7Rm4Q=="}' 2026-02-19T13:20:24.823 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9cd9be71-cc4f-4918-bf1f-e12be610e793 -i td/test-erasure-eio/3/new.json 2026-02-19T13:20:25.046 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:20:25.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:20:25.059 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAYDpdpIPofMRAA5SS4fMnpADK/Vy38I7Rm4Q== --osd-uuid 9cd9be71-cc4f-4918-bf1f-e12be610e793 2026-02-19T13:20:25.074 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:25.075+0000 7f6d93d74900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:25.076 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:25.076+0000 7f6d93d74900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:25.077 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:25.077+0000 7f6d93d74900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:25.077 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:25.077+0000 7f6d93d74900 -1 bdev(0x55c49401ec00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:25.077 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:25.077+0000 7f6d93d74900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:20:25.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:20:25.486 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:25.487 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:20:25.488 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:20:25.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:25.790 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:20:25.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:20:25.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:25.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:25.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:25.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:25.805 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:25.805+0000 7f7204c7c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:25.806 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:25.806+0000 7f7204c7c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:25.807 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:25.807+0000 7f7204c7c900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:26.011 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:20:26.021 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:26.022+0000 7f7204c7c900 -1 Falling back to public interface 2026-02-19T13:20:26.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:26.372 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:26.372+0000 7f7204c7c900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:20:27.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:27.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:27.236 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:27.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:27.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:27.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:20:27.460 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 27 up_thru 28 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/2698143951,v1:127.0.0.1:6827/2698143951] [v2:127.0.0.1:6828/2698143951,v1:127.0.0.1:6829/2698143951] exists,up 9cd9be71-cc4f-4918-bf1f-e12be610e793 2026-02-19T13:20:27.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:27.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:27.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:27.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:20:27.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:20:27.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:20:27.462 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:27.462 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:27.463 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:27.463 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:20:27.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:20:27.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:20:27.521 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:20:27.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:20:27.531 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:20:17.946+0000 7fc61ef91900 0 load: jerasure load: lrc 2026-02-19T13:20:27.532 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:290: TEST_rados_get_subread_eio_shard_1: local poolname=pool-jerasure 2026-02-19T13:20:27.532 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:291: TEST_rados_get_subread_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T13:20:27.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:20:27.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:20:27.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T13:20:27.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:20:27.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T13:20:27.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:20:27.533 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T13:20:27.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:20:27.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:20:28.175 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:20:28.186 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:20:29.187 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:20:29.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:20:29.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:20:29.188 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:20:29.189 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:20:29.189 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:20:29.189 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:20:29.189 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:20:29.189 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:20:29.189 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:20:29.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:20:29.269 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:20:29.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:20:29.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:20:29.270 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:29.270 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:29.488 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:29.488 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:29.488 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:20:29.489 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:20:29.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:29.489 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:29.489 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:29.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T13:20:29.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T13:20:29.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T13:20:29.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:29.570 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:29.649 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574853 2026-02-19T13:20:29.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574853 2026-02-19T13:20:29.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853' 2026-02-19T13:20:29.650 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:29.650 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:29.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T13:20:29.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T13:20:29.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411331' 2026-02-19T13:20:29.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:29.734 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:20:29.811 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=115964116995 2026-02-19T13:20:29.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 115964116995 2026-02-19T13:20:29.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574853 2-77309411331 3-115964116995' 2026-02-19T13:20:29.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:29.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T13:20:29.812 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:29.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:29.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-19T13:20:29.813 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:29.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-19T13:20:29.814 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803781 2026-02-19T13:20:29.815 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T13:20:29.815 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:30.057 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803781 2026-02-19T13:20:30.057 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:31.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:31.058 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:31.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-19T13:20:31.284 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:31.285 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574853 2026-02-19T13:20:31.285 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:31.285 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:31.286 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574853 2026-02-19T13:20:31.286 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:31.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574853 2026-02-19T13:20:31.287 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574853' 2026-02-19T13:20:31.288 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574853 2026-02-19T13:20:31.288 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:31.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574853 -lt 55834574853 2026-02-19T13:20:31.509 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:31.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T13:20:31.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:31.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:31.511 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411331 2026-02-19T13:20:31.511 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:31.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411331 2026-02-19T13:20:31.512 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T13:20:31.512 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411331 2026-02-19T13:20:31.512 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:31.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T13:20:31.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:31.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-115964116995 2026-02-19T13:20:31.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:31.737 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:20:31.737 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-115964116995 2026-02-19T13:20:31.737 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:31.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=115964116995 2026-02-19T13:20:31.739 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 115964116995 2026-02-19T13:20:31.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 115964116995' 2026-02-19T13:20:31.739 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:20:31.962 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 115964116995 -lt 115964116995 2026-02-19T13:20:31.963 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:20:31.963 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:20:31.963 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:20:32.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:20:32.262 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:20:32.262 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:20:32.262 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:20:32.263 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:20:32.263 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:20:32.263 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:20:32.263 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:20:32.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:20:32.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:20:32.479 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:20:32.480 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:20:32.777 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:293: TEST_rados_get_subread_eio_shard_1: local shard_id=1 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:294: TEST_rados_get_subread_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-19T13:20:32.778 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-65583-1 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-65583-1 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-65583-1 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:32.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:20:32.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:32.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:20:32.780 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-65583-1 td/test-erasure-eio/ORIGINAL 2026-02-19T13:20:32.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-65583-1 td/test-erasure-eio 1 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-65583-1 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:20:32.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:20:32.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:20:32.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:20:32.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-65583-1 2026-02-19T13:20:32.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:20:32.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-1 2026-02-19T13:20:32.797 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-1 2026-02-19T13:20:32.797 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:20:33.020 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:20:33.021 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:33.021 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:20:33.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:20:33.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T13:20:33.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:20:33.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:20:33.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:20:33.021 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T13:20:33.023 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:20:33.023 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:33.024 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:20:33.025 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:20:33.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:20:33.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:20:33.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:20:33.089 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:20:33.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:20:33.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:20:33.090 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:33.090 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:33.090 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:33.090 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:20:33.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:20:33.090 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-65583-1 1 2026-02-19T13:20:33.152 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-65583-1 2026-02-19T13:20:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:20:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:20:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-65583-1 2026-02-19T13:20:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:20:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:20:33.153 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-65583-1 td/test-erasure-eio/COPY 2026-02-19T13:20:33.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:20:33.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:20:33.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-19T13:20:33.171 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-19T13:20:33.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-19T13:20:33.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-65583-1 td/test-erasure-eio 2 2026-02-19T13:20:33.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:20:33.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:20:33.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:20:33.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:20:33.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-65583-1 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=2 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-65583-1 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-1 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-1 2026-02-19T13:20:33.175 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:20:33.401 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:20:33.401 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:33.401 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:20:33.401 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:20:33.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T13:20:33.401 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:20:33.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=2 2026-02-19T13:20:33.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:20:33.402 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/2/type 2026-02-19T13:20:33.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 2 bluestore_debug_inject_read_err true 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=2 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.2 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:33.404 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:33.405 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:33.405 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.2.asok 2026-02-19T13:20:33.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.2.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:20:33.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:20:33.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:20:33.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.2 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.2 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.2 ']' 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.2.asok 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:20:33.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.2.asok injectdataerr pool-jerasure obj-eio-65583-1 2 2026-02-19T13:20:33.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T13:20:33.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-65583-1 2026-02-19T13:20:33.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:20:33.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:20:33.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-65583-1 2026-02-19T13:20:33.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:20:33.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:20:33.536 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-65583-1 td/test-erasure-eio/COPY 2026-02-19T13:20:33.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:20:33.553 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:20:33.554 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:20:33.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:295: TEST_rados_get_subread_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:20:33.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:20:33.555 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:20:33.820 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:20:33.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:20:34.096 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:20:34.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:20:34.108 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:20:34.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:20:34.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:20:34.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:20:34.109 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:20:34.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:20:34.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:20:34.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:20:34.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:20:34.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:20:34.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:20:34.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:20:34.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:20:34.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:20:34.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:20:34.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:20:34.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:20:34.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:20:34.221 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:20:34.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:20:34.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:20:34.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:20:34.231 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:20:34.231 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:34.231 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:34.231 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:20:34.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:20:34.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:20:34.233 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:20:34.234 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:20:34.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:20:34.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:20:34.235 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:20:34.235 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:20:34.236 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:20:34.236 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:20:34.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:20:34.237 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:20:34.237 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:20:34.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:20:34.238 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:20:34.239 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:20:34.239 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:20:34.239 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:20:34.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:20:34.240 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:20:34.242 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:20:34.242 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:34.242 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:34.242 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:20:34.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:20:34.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:20:34.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:20:34.244 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:20:34.244 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:34.244 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:34.244 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:20:34.245 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:20:34.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:20:34.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:20:34.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:20:34.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:34.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:34.267 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:34.267 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:34.268 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:34.268 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:34.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:20:34.291 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:20:34.292 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:20:34.292 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:20:34.292 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:20:34.293 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:20:34.293 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:20:34.294 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:20:34.294 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:20:34.294 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:20:34.294 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:34.294 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:34.294 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:34.294 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:20:34.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:20:34.295 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:20:34.355 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:20:34.355 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:20:34.355 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:20:34.355 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:20:34.355 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:20:34.355 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:20:34.355 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:20:34.356 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:20:34.356 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:34.356 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:34.356 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:34.356 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:20:34.357 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:20:34.357 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:20:34.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:20:34.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:20:34.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:20:34.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:20:34.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:20:34.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:20:34.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:20:34.528 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:20:34.528 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:34.528 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:34.528 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:34.528 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:34.529 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:34.529 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:34.529 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:20:34.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:20:34.545 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:20:34.546 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:20:34.649 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:20:34.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:20:35.662 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:20:35.662 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:20:35.662 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:20:35.663 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:35.663 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:35.663 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:35.664 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:20:35.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:20:35.664 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:20:35.717 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:20:35.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:20:35.724 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:20:34.285+0000 7f7083649dc0 0 load: jerasure load: lrc 2026-02-19T13:20:35.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_subread_missing td/test-erasure-eio 2026-02-19T13:20:35.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:302: TEST_rados_get_subread_missing: local dir=td/test-erasure-eio 2026-02-19T13:20:35.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:303: TEST_rados_get_subread_missing: setup_osds 4 2026-02-19T13:20:35.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T13:20:35.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:20:35.725 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T13:20:35.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:35.728 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:35.729 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:20:35.730 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:35.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=a5079c1c-cd0f-4898-aaef-6e3d056cc77a 2026-02-19T13:20:35.731 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 a5079c1c-cd0f-4898-aaef-6e3d056cc77a' 2026-02-19T13:20:35.731 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 a5079c1c-cd0f-4898-aaef-6e3d056cc77a 2026-02-19T13:20:35.731 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:35.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAjDpdpiAhOLBAA/7eoYJqViJVWQkg16POUsg== 2026-02-19T13:20:35.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAjDpdpiAhOLBAA/7eoYJqViJVWQkg16POUsg=="}' 2026-02-19T13:20:35.743 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new a5079c1c-cd0f-4898-aaef-6e3d056cc77a -i td/test-erasure-eio/0/new.json 2026-02-19T13:20:35.964 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:35.976 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:20:35.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAjDpdpiAhOLBAA/7eoYJqViJVWQkg16POUsg== --osd-uuid a5079c1c-cd0f-4898-aaef-6e3d056cc77a 2026-02-19T13:20:35.992 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:35.993+0000 7f4ea6671900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:35.994 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:35.994+0000 7f4ea6671900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:35.994 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:35.995+0000 7f4ea6671900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:35.995 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:35.995+0000 7f4ea6671900 -1 bdev(0x56002192ac00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:35.995 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:35.995+0000 7f4ea6671900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:20:36.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:20:36.445 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:36.445 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:20:36.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:20:36.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:36.745 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:20:36.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:20:36.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:36.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:36.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:36.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:36.760 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:36.760+0000 7f1483512900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:36.761 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:36.761+0000 7f1483512900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:36.762 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:36.762+0000 7f1483512900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:36.962 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:36.962+0000 7f1483512900 -1 Falling back to public interface 2026-02-19T13:20:36.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:20:36.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:36.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:20:36.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:36.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:36.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:36.965 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:36.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:36.965 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:36.966 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:20:37.190 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:37.301 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:37.301+0000 7f1483512900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:20:38.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:38.191 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:38.192 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:38.193 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:20:38.416 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/45015414,v1:127.0.0.1:6803/45015414] [v2:127.0.0.1:6804/45015414,v1:127.0.0.1:6805/45015414] exists,up a5079c1c-cd0f-4898-aaef-6e3d056cc77a 2026-02-19T13:20:38.420 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:20:38.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:38.422 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:38.423 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:38.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:38.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:38.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:38.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:38.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:20:38.424 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:38.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=049230ae-9a53-4a59-b54d-0592f4d8cd52 2026-02-19T13:20:38.424 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 049230ae-9a53-4a59-b54d-0592f4d8cd52' 2026-02-19T13:20:38.425 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 049230ae-9a53-4a59-b54d-0592f4d8cd52 2026-02-19T13:20:38.425 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:38.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQAmDpdpJXnHGRAAc2D44vxFSz/rmS6mPCU50g== 2026-02-19T13:20:38.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQAmDpdpJXnHGRAAc2D44vxFSz/rmS6mPCU50g=="}' 2026-02-19T13:20:38.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 049230ae-9a53-4a59-b54d-0592f4d8cd52 -i td/test-erasure-eio/1/new.json 2026-02-19T13:20:38.649 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:38.660 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:20:38.661 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQAmDpdpJXnHGRAAc2D44vxFSz/rmS6mPCU50g== --osd-uuid 049230ae-9a53-4a59-b54d-0592f4d8cd52 2026-02-19T13:20:38.677 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:38.677+0000 7ff8c38fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:38.678 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:38.679+0000 7ff8c38fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:38.679 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:38.680+0000 7ff8c38fe900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:38.680 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:38.680+0000 7ff8c38fe900 -1 bdev(0x55cc71632c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:38.680 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:38.680+0000 7ff8c38fe900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:20:39.103 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:20:39.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:39.104 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:20:39.105 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:20:39.105 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:39.406 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:20:39.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:20:39.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:39.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:39.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:39.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:39.420 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:39.421+0000 7f2df8661900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:39.422 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:39.422+0000 7f2df8661900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:39.422 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:39.423+0000 7f2df8661900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:39.626 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:20:39.626 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:39.626 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:20:39.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:39.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:39.627 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:39.627 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:39.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:39.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:39.628 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:20:39.640 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:39.640+0000 7f2df8661900 -1 Falling back to public interface 2026-02-19T13:20:39.846 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:40.002 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:40.002+0000 7f2df8661900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:20:40.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:40.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:40.847 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:40.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:40.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:40.847 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:20:41.069 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/2495616226,v1:127.0.0.1:6811/2495616226] [v2:127.0.0.1:6812/2495616226,v1:127.0.0.1:6813/2495616226] exists,up 049230ae-9a53-4a59-b54d-0592f4d8cd52 2026-02-19T13:20:41.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:41.070 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:41.071 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:20:41.072 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:41.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=7459f276-6db4-488e-84a8-02d00d2ae319 2026-02-19T13:20:41.073 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 7459f276-6db4-488e-84a8-02d00d2ae319' 2026-02-19T13:20:41.073 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 7459f276-6db4-488e-84a8-02d00d2ae319 2026-02-19T13:20:41.073 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:41.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQApDpdpnRIYBRAAX2Alqr/4FPayft5m3PKNTw== 2026-02-19T13:20:41.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQApDpdpnRIYBRAAX2Alqr/4FPayft5m3PKNTw=="}' 2026-02-19T13:20:41.084 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 7459f276-6db4-488e-84a8-02d00d2ae319 -i td/test-erasure-eio/2/new.json 2026-02-19T13:20:41.303 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:20:41.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:20:41.315 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQApDpdpnRIYBRAAX2Alqr/4FPayft5m3PKNTw== --osd-uuid 7459f276-6db4-488e-84a8-02d00d2ae319 2026-02-19T13:20:41.330 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:41.332+0000 7ff07004f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:41.332 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:41.333+0000 7ff07004f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:41.333 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:41.334+0000 7ff07004f900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:41.333 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:41.334+0000 7ff07004f900 -1 bdev(0x56515da64c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:41.333 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:41.334+0000 7ff07004f900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:20:41.754 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:20:41.754 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:41.754 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:20:41.755 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:20:41.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:42.052 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:20:42.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:20:42.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:42.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:42.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:42.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:42.067 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:42.068+0000 7f6805272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:42.068 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:42.070+0000 7f6805272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:42.069 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:42.071+0000 7f6805272900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:42.250 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:42.251+0000 7f6805272900 -1 Falling back to public interface 2026-02-19T13:20:42.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:20:42.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:42.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:20:42.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:42.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:42.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:42.273 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:42.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:42.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:42.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:20:42.494 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:42.662 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:42.663+0000 7f6805272900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:20:43.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:43.495 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:43.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:43.496 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:43.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:43.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:20:43.720 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/2445468940,v1:127.0.0.1:6819/2445468940] [v2:127.0.0.1:6820/2445468940,v1:127.0.0.1:6821/2445468940] exists,up 7459f276-6db4-488e-84a8-02d00d2ae319 2026-02-19T13:20:43.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:43.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:43.725 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:43.726 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:43.727 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:43.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:43.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:43.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:20:43.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:20:43.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:20:43.728 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:20:43.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=25b7751f-14de-4b1b-91c4-080664b3db80 2026-02-19T13:20:43.728 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 25b7751f-14de-4b1b-91c4-080664b3db80' 2026-02-19T13:20:43.729 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 25b7751f-14de-4b1b-91c4-080664b3db80 2026-02-19T13:20:43.729 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:20:43.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQArDpdpRuTfKxAAfsx0bDmAsAogaOu7rMCwRA== 2026-02-19T13:20:43.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQArDpdpRuTfKxAAfsx0bDmAsAogaOu7rMCwRA=="}' 2026-02-19T13:20:43.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 25b7751f-14de-4b1b-91c4-080664b3db80 -i td/test-erasure-eio/3/new.json 2026-02-19T13:20:43.956 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:20:43.967 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:20:43.968 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQArDpdpRuTfKxAAfsx0bDmAsAogaOu7rMCwRA== --osd-uuid 25b7751f-14de-4b1b-91c4-080664b3db80 2026-02-19T13:20:43.984 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:43.986+0000 7ff86c268900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:43.986 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:43.987+0000 7ff86c268900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:43.987 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:43.988+0000 7ff86c268900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:43.987 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:43.988+0000 7ff86c268900 -1 bdev(0x55b5b1830c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:20:43.987 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:43.988+0000 7ff86c268900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:20:44.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:20:44.431 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:20:44.432 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:20:44.432 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:20:44.433 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:20:44.734 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:20:44.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:20:44.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:44.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:20:44.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:20:44.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:20:44.749 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:44.751+0000 7f3249fdf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:44.751 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:44.752+0000 7f3249fdf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:44.751 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:44.753+0000 7f3249fdf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:44.953 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:44.955+0000 7f3249fdf900 -1 Falling back to public interface 2026-02-19T13:20:44.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:20:44.956 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:44.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:20:44.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:44.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:44.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:44.957 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:44.957 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:20:44.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:44.958 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:20:45.176 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:45.316 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:45.317+0000 7f3249fdf900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:20:46.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:46.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:46.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:46.178 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:20:46.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:46.179 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:20:46.402 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3951135271,v1:127.0.0.1:6827/3951135271] [v2:127.0.0.1:6828/3951135271,v1:127.0.0.1:6829/3951135271] exists,up 25b7751f-14de-4b1b-91c4-080664b3db80 2026-02-19T13:20:46.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:46.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:46.404 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:46.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:20:46.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:20:46.404 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:20:46.404 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:20:46.404 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:46.405 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:46.405 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:20:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:20:46.405 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:20:46.461 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:20:46.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:20:46.470 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:20:36.966+0000 7f1483512900 0 load: jerasure load: lrc 2026-02-19T13:20:46.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:305: TEST_rados_get_subread_missing: local poolname=pool-jerasure 2026-02-19T13:20:46.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:306: TEST_rados_get_subread_missing: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T13:20:46.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:20:46.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:20:46.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T13:20:46.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:20:46.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T13:20:46.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:20:46.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T13:20:46.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:20:46.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:20:47.149 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:20:47.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:20:48.163 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:20:48.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:20:48.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:20:48.164 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:20:48.165 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:20:48.165 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:20:48.165 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:20:48.165 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:20:48.165 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:20:48.166 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:20:48.246 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:20:48.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:20:48.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:20:48.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:20:48.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:48.247 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:48.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:48.470 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:48.470 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:20:48.470 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:20:48.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:48.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:48.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:48.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T13:20:48.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T13:20:48.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T13:20:48.552 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:48.552 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:48.634 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-19T13:20:48.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-19T13:20:48.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-19T13:20:48.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:48.635 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:48.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T13:20:48.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T13:20:48.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411331' 2026-02-19T13:20:48.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:48.713 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:20:48.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T13:20:48.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T13:20:48.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411331 3-111669149699' 2026-02-19T13:20:48.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:48.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T13:20:48.796 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:48.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:48.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-19T13:20:48.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:48.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-19T13:20:48.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T13:20:48.798 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803781 2026-02-19T13:20:48.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:49.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803781 2026-02-19T13:20:49.018 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:50.019 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:50.020 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:50.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-19T13:20:50.243 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:50.244 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-19T13:20:50.244 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:50.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:50.245 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574852 2026-02-19T13:20:50.246 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:50.246 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574852 2026-02-19T13:20:50.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-19T13:20:50.247 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574852 2026-02-19T13:20:50.247 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:50.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-19T13:20:50.473 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:50.474 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T13:20:50.474 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:50.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:50.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411331 2026-02-19T13:20:50.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:50.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411331 2026-02-19T13:20:50.476 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T13:20:50.477 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411331 2026-02-19T13:20:50.477 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:50.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T13:20:50.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:50.702 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T13:20:50.702 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:50.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:20:50.703 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149699 2026-02-19T13:20:50.703 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:50.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149699 2026-02-19T13:20:50.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T13:20:50.705 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149699 2026-02-19T13:20:50.705 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:20:50.925 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-19T13:20:50.926 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:20:50.927 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:20:50.927 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:20:51.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:20:51.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:20:51.226 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:20:51.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:20:51.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:20:51.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:20:51.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:20:51.227 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:20:51.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:20:51.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:20:51.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:20:51.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:20:51.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:20:51.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:20:51.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:20:51.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:308: TEST_rados_get_subread_missing: local shard_id=1 2026-02-19T13:20:51.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:309: TEST_rados_get_subread_missing: rados_put_get_data remove td/test-erasure-eio 1 2026-02-19T13:20:51.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=remove 2026-02-19T13:20:51.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T13:20:51.749 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg= 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-remove-65583-1 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-remove-65583-1 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-remove-65583-1 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:51.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:20:51.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:51.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:20:51.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:51.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:20:51.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:20:51.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:20:51.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-remove-65583-1 td/test-erasure-eio/ORIGINAL 2026-02-19T13:20:51.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-65583-1 td/test-erasure-eio 1 2026-02-19T13:20:51.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-19T13:20:51.768 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-19T13:20:51.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-19T13:20:51.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-19T13:20:51.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-19T13:20:51.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-19T13:20:51.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-65583-1 2026-02-19T13:20:51.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-19T13:20:51.769 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-19T13:20:51.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-19T13:20:51.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=1 2026-02-19T13:20:51.770 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-19T13:20:51.770 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-65583-1 2026-02-19T13:20:51.770 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:20:51.770 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-65583-1 2026-02-19T13:20:51.770 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-65583-1 2026-02-19T13:20:51.771 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:20:51.991 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:20:51.992 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:51.992 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:20:51.992 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:20:51.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-19T13:20:51.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-19T13:20:51.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=1 2026-02-19T13:20:51.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 1 obj-remove-65583-1 remove 2026-02-19T13:20:51.992 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=1 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 1 obj-remove-65583-1 remove 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=1 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.1 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:20:51.993 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:20:51.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:20:51.994 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:20:52.298 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:20:52.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 1 obj-remove-65583-1 remove 2026-02-19T13:20:52.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:20:52.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:20:52.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=1 2026-02-19T13:20:52.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:20:52.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/1 2026-02-19T13:20:52.299 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/1 obj-remove-65583-1 remove 2026-02-19T13:20:52.418 INFO:tasks.workunit.client.0.trial111.stdout:remove 1#2:9cc902ae:::obj-remove-65583-1:head# 2026-02-19T13:20:52.467 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 1 2026-02-19T13:20:52.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:52.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:20:52.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=1 2026-02-19T13:20:52.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:20:52.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:20:52.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:52.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:52.468 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:52.469 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:20:52.470 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:20:52.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.1 2026-02-19T13:20:52.471 INFO:tasks.workunit.client.0.trial111.stderr:start osd.1 2026-02-19T13:20:52.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:52.471 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/1/whoami 2026-02-19T13:20:52.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 1 = 1 ']' 2026-02-19T13:20:52.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:20:52.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:20:52.472 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:20:52.481 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:52.482+0000 7f8c6a025900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:52.483 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:52.483+0000 7f8c6a025900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:52.484 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:52.484+0000 7f8c6a025900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:52.679 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:52.679+0000 7f8c6a025900 -1 Falling back to public interface 2026-02-19T13:20:52.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 1 2026-02-19T13:20:52.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:52.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:20:52.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:52.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:52.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:52.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:52.691 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:20:52.692 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:52.692 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:20:52.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:53.061 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:53.061+0000 7f8c6a025900 -1 osd.1 33 log_to_monitors true 2026-02-19T13:20:53.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:53.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:53.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:53.914 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:53.914 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:53.914 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:20:54.140 INFO:tasks.workunit.client.0.trial111.stderr:osd.1 up in weight 1 up_from 37 up_thru 37 down_at 34 last_clean_interval [13,33) [v2:127.0.0.1:6810/3331163100,v1:127.0.0.1:6811/3331163100] [v2:127.0.0.1:6812/3331163100,v1:127.0.0.1:6813/3331163100] exists,up 049230ae-9a53-4a59-b54d-0592f4d8cd52 2026-02-19T13:20:54.140 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:54.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:54.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:54.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:20:54.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:20:54.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:20:54.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:20:54.141 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:20:54.142 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:20:54.142 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:20:54.142 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:20:54.142 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:20:54.142 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:20:54.217 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:20:54.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:20:54.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:20:54.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:20:54.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:20:54.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:20:54.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:20:54.437 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:54.437 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:20:54.437 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:20:54.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:20:54.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:54.437 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:20:54.515 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-19T13:20:54.515 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-19T13:20:54.515 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784' 2026-02-19T13:20:54.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:54.516 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:20:54.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789955 2026-02-19T13:20:54.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789955 2026-02-19T13:20:54.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-158913789955' 2026-02-19T13:20:54.598 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:54.598 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:20:54.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411335 2026-02-19T13:20:54.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411335 2026-02-19T13:20:54.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-158913789955 2-77309411335' 2026-02-19T13:20:54.675 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:20:54.675 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:20:54.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T13:20:54.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T13:20:54.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-158913789955 2-77309411335 3-111669149702' 2026-02-19T13:20:54.755 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:54.755 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-19T13:20:54.756 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:54.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:20:54.757 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803784 2026-02-19T13:20:54.757 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:54.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803784 2026-02-19T13:20:54.759 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-19T13:20:54.759 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803784 2026-02-19T13:20:54.759 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:54.977 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803784 2026-02-19T13:20:54.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:20:55.978 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:20:55.979 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:20:56.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-19T13:20:56.203 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:56.203 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-158913789955 2026-02-19T13:20:56.203 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:56.204 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:20:56.205 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-158913789955 2026-02-19T13:20:56.205 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:56.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789955 2026-02-19T13:20:56.206 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 158913789955' 2026-02-19T13:20:56.206 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 158913789955 2026-02-19T13:20:56.207 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:20:56.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789955 -lt 158913789955 2026-02-19T13:20:56.427 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:56.428 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411335 2026-02-19T13:20:56.428 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:56.428 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:20:56.429 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411335 2026-02-19T13:20:56.429 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:56.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411335 2026-02-19T13:20:56.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411335' 2026-02-19T13:20:56.430 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 77309411335 2026-02-19T13:20:56.431 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:20:56.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411335 -lt 77309411335 2026-02-19T13:20:56.656 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:20:56.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T13:20:56.656 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:20:56.657 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:20:56.658 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149702 2026-02-19T13:20:56.658 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:20:56.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149702 2026-02-19T13:20:56.659 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T13:20:56.659 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149702 2026-02-19T13:20:56.660 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:20:56.889 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149702 -lt 111669149702 2026-02-19T13:20:56.889 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:20:56.890 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:20:56.890 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:20:57.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:20:57.189 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:20:57.190 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:20:57.190 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:20:57.190 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:20:57.191 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:20:57.191 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:20:57.191 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:20:57.406 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:20:57.406 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:20:57.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:20:57.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:20:57.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:20:57.717 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:20:57.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:20:57.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-65583-1 2026-02-19T13:20:57.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:20:57.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:20:57.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-65583-1 2026-02-19T13:20:57.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:20:57.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:20:57.718 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-65583-1 td/test-erasure-eio/COPY 2026-02-19T13:20:57.734 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:20:57.735 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:20:57.736 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' '' = recovery ']' 2026-02-19T13:20:57.736 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: expr 1 + 1 2026-02-19T13:20:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:181: rados_put_get_data: local another_shard_id=2 2026-02-19T13:20:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:182: rados_put_get_data: inject_remove ec data pool-jerasure obj-remove-65583-1 td/test-erasure-eio 2 2026-02-19T13:20:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:127: inject_remove: local pooltype=ec 2026-02-19T13:20:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:128: inject_remove: shift 2026-02-19T13:20:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:129: inject_remove: local which=data 2026-02-19T13:20:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:130: inject_remove: shift 2026-02-19T13:20:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:131: inject_remove: local poolname=pool-jerasure 2026-02-19T13:20:57.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:132: inject_remove: shift 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:133: inject_remove: local objname=obj-remove-65583-1 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:134: inject_remove: shift 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:135: inject_remove: local dir=td/test-erasure-eio 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:136: inject_remove: shift 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:137: inject_remove: local shard_id=2 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:138: inject_remove: shift 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: get_osds pool-jerasure obj-remove-65583-1 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-remove-65583-1 2026-02-19T13:20:57.740 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-remove-65583-1 2026-02-19T13:20:57.741 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:20:57.962 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: initial_osds=('3' '1' '2') 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:140: inject_remove: local -a initial_osds 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:141: inject_remove: local osd_id=2 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:142: inject_remove: objectstore_tool td/test-erasure-eio 2 obj-remove-65583-1 remove 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1288: objectstore_tool: local dir=td/test-erasure-eio 2026-02-19T13:20:57.963 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1289: objectstore_tool: shift 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1290: objectstore_tool: local id=2 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1291: objectstore_tool: shift 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1293: objectstore_tool: _objectstore_tool_nowait td/test-erasure-eio 2 obj-remove-65583-1 remove 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1260: _objectstore_tool_nowait: local dir=td/test-erasure-eio 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1261: _objectstore_tool_nowait: shift 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1262: _objectstore_tool_nowait: local id=2 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1263: _objectstore_tool_nowait: shift 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1265: _objectstore_tool_nowait: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:20:57.964 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:20:58.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:20:58.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1267: _objectstore_tool_nowait: _objectstore_tool_nodown td/test-erasure-eio 2 obj-remove-65583-1 remove 2026-02-19T13:20:58.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1248: _objectstore_tool_nodown: local dir=td/test-erasure-eio 2026-02-19T13:20:58.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1249: _objectstore_tool_nodown: shift 2026-02-19T13:20:58.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1250: _objectstore_tool_nodown: local id=2 2026-02-19T13:20:58.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1251: _objectstore_tool_nodown: shift 2026-02-19T13:20:58.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1252: _objectstore_tool_nodown: local osd_data=td/test-erasure-eio/2 2026-02-19T13:20:58.069 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1255: _objectstore_tool_nodown: ceph-objectstore-tool --data-path td/test-erasure-eio/2 obj-remove-65583-1 remove 2026-02-19T13:20:58.197 INFO:tasks.workunit.client.0.trial111.stdout:remove 2#2:9cc902ae:::obj-remove-65583-1:head# 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1268: _objectstore_tool_nowait: activate_osd td/test-erasure-eio 2 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:20:58.252 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:20:58.253 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:20:58.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:20:58.254 INFO:tasks.workunit.client.0.trial111.stderr:start osd.2 2026-02-19T13:20:58.254 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:20:58.254 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:20:58.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:20:58.255 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:20:58.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:20:58.256 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:20:58.269 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:58.269+0000 7fda14e9a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:58.270 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:58.271+0000 7fda14e9a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:58.271 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:58.272+0000 7fda14e9a900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:20:58.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:20:58.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:20:58.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:20:58.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:20:58.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:20:58.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:58.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:20:58.477 INFO:tasks.workunit.client.0.trial111.stderr:0 2026-02-19T13:20:58.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:58.478 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:20:58.484 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:58.484+0000 7fda14e9a900 -1 Falling back to public interface 2026-02-19T13:20:58.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:20:58.887 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:20:58.888+0000 7fda14e9a900 -1 osd.2 40 log_to_monitors true 2026-02-19T13:20:59.702 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:20:59.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:20:59.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:20:59.703 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:20:59.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:20:59.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:20:59.923 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 up in weight 1 up_from 43 up_thru 34 down_at 40 last_clean_interval [18,40) [v2:127.0.0.1:6818/2520598442,v1:127.0.0.1:6819/2520598442] [v2:127.0.0.1:6820/2520598442,v1:127.0.0.1:6821/2520598442] exists,up 7459f276-6db4-488e-84a8-02d00d2ae319 2026-02-19T13:20:59.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:20:59.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:20:59.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:20:59.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1294: objectstore_tool: wait_for_clean 2026-02-19T13:20:59.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:20:59.923 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:20:59.924 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:20:59.924 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:20:59.925 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:20:59.925 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:20:59.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:20:59.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:20:59.925 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:21:00.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:21:00.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:21:00.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:21:00.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:21:00.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:21:00.002 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:21:00.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:21:00.221 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:00.221 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:21:00.221 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:21:00.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:21:00.222 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:00.222 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:21:00.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803787 2026-02-19T13:21:00.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803787 2026-02-19T13:21:00.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787' 2026-02-19T13:21:00.300 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:00.300 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:21:00.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=158913789958 2026-02-19T13:21:00.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 158913789958 2026-02-19T13:21:00.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-158913789958' 2026-02-19T13:21:00.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:00.379 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:21:00.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-19T13:21:00.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-19T13:21:00.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-158913789958 2-184683593731' 2026-02-19T13:21:00.457 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:00.457 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:21:00.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149705 2026-02-19T13:21:00.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149705 2026-02-19T13:21:00.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803787 1-158913789958 2-184683593731 3-111669149705' 2026-02-19T13:21:00.537 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:00.538 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803787 2026-02-19T13:21:00.538 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:00.538 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:21:00.538 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803787 2026-02-19T13:21:00.539 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:00.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803787 2026-02-19T13:21:00.540 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803787' 2026-02-19T13:21:00.540 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.0 seq 25769803787 2026-02-19T13:21:00.540 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:00.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803785 -lt 25769803787 2026-02-19T13:21:00.761 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:21:01.762 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:21:01.763 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:01.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803787 -lt 25769803787 2026-02-19T13:21:01.986 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:01.987 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-158913789958 2026-02-19T13:21:01.987 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:01.988 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:21:01.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-158913789958 2026-02-19T13:21:01.988 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:01.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=158913789958 2026-02-19T13:21:01.990 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 158913789958' 2026-02-19T13:21:01.990 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.1 seq 158913789958 2026-02-19T13:21:01.990 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:21:02.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 158913789958 -lt 158913789958 2026-02-19T13:21:02.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:02.213 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-184683593731 2026-02-19T13:21:02.213 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:02.213 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:21:02.214 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-184683593731 2026-02-19T13:21:02.214 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:02.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593731 2026-02-19T13:21:02.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-19T13:21:02.215 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.2 seq 184683593731 2026-02-19T13:21:02.216 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:21:02.437 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-19T13:21:02.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:02.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149705 2026-02-19T13:21:02.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:02.439 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:21:02.439 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149705 2026-02-19T13:21:02.440 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:02.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149705 2026-02-19T13:21:02.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149705' 2026-02-19T13:21:02.441 INFO:tasks.workunit.client.0.trial111.stderr:waiting osd.3 seq 111669149705 2026-02-19T13:21:02.441 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:21:02.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149705 -lt 111669149705 2026-02-19T13:21:02.681 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:21:02.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:02.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:02.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:21:02.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:21:02.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:21:02.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:21:02.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:21:02.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:21:02.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:21:02.981 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:21:03.198 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:21:03.198 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:21:03.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:03.199 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:03.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:21:03.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:21:03.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:21:03.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:183: rados_put_get_data: '[' 1 -eq 1 -a 2 -eq 2 ']' 2026-02-19T13:21:03.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:188: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-remove-65583-1 2026-02-19T13:21:03.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:21:03.496 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:21:03.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-remove-65583-1 2026-02-19T13:21:03.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:21:03.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:21:03.497 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-remove-65583-1 td/test-erasure-eio/COPY 2026-02-19T13:21:03.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:21:03.515 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:21:03.516 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:194: rados_put_get_data: rm td/test-erasure-eio/ORIGINAL 2026-02-19T13:21:03.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:310: TEST_rados_get_subread_missing: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:21:03.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:21:03.517 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:21:03.799 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:21:03.810 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:21:04.076 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:21:04.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:21:04.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:21:04.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:21:04.088 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:21:04.088 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:21:04.088 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:21:04.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:21:04.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:21:04.089 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:21:04.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:21:04.196 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:21:04.196 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:21:04.196 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:21:04.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:21:04.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:21:04.198 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:21:04.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:04.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:21:04.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:21:04.200 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:04.200 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:21:04.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:21:04.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:21:04.212 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:21:04.212 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:04.212 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:04.212 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:21:04.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:21:04.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:21:04.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:21:04.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:21:04.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:21:04.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:21:04.214 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:21:04.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:21:04.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:21:04.215 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:21:04.215 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:21:04.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:21:04.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:21:04.215 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:21:04.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:21:04.216 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:21:04.216 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:21:04.217 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:21:04.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:21:04.218 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:21:04.218 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:21:04.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:04.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:21:04.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:21:04.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:04.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:21:04.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:21:04.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:21:04.222 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:21:04.223 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:04.223 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:04.223 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:21:04.223 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:21:04.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:21:04.224 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:21:04.225 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:21:04.225 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:04.225 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:04.225 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:21:04.226 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:21:04.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:21:04.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:21:04.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:21:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:21:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:21:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:21:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:21:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:21:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:21:04.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:21:04.247 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:21:04.247 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:04.248 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:04.248 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:04.248 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:04.248 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:04.248 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:04.248 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:21:04.272 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:21:04.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:21:04.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:21:04.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:21:04.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:21:04.273 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:21:04.275 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:21:04.336 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:21:04.336 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:21:04.336 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:21:04.336 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:21:04.336 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:21:04.337 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:21:04.338 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:21:04.338 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:21:04.338 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:04.338 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:04.338 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:04.338 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:21:04.338 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:21:04.338 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:21:04.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:21:04.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:21:04.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:21:04.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:21:04.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:21:04.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:21:04.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:21:04.509 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:21:04.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:04.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:04.510 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:04.510 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:04.510 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:04.510 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:04.511 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:21:04.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:21:04.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:21:04.527 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:21:04.628 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:21:04.638 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:21:05.640 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:21:05.640 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:21:05.640 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:21:05.641 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:05.641 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:05.642 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:05.642 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:21:05.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:21:05.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:21:05.694 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:21:05.701 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:21:05.702 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:21:04.265+0000 7f353b255dc0 0 load: jerasure load: lrc 2026-02-19T13:21:05.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_0 td/test-erasure-eio 2026-02-19T13:21:05.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:350: TEST_rados_get_with_subreadall_eio_shard_0: local dir=td/test-erasure-eio 2026-02-19T13:21:05.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:351: TEST_rados_get_with_subreadall_eio_shard_0: local shard_id=0 2026-02-19T13:21:05.703 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:353: TEST_rados_get_with_subreadall_eio_shard_0: setup_osds 4 2026-02-19T13:21:05.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T13:21:05.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:21:05.704 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T13:21:05.708 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T13:21:05.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:21:05.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:21:05.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:05.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:21:05.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:21:05.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:05.709 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:05.710 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=878937b3-bf51-45ae-8ada-423109d5e77a 2026-02-19T13:21:05.711 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 878937b3-bf51-45ae-8ada-423109d5e77a 2026-02-19T13:21:05.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 878937b3-bf51-45ae-8ada-423109d5e77a' 2026-02-19T13:21:05.712 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:21:05.719 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBBDpdp6d7qKhAAJs3umJZF5pmZ1Ro4z2a9gA== 2026-02-19T13:21:05.720 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBBDpdp6d7qKhAAJs3umJZF5pmZ1Ro4z2a9gA=="}' 2026-02-19T13:21:05.720 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 878937b3-bf51-45ae-8ada-423109d5e77a -i td/test-erasure-eio/0/new.json 2026-02-19T13:21:05.937 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:05.948 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:21:05.949 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBBDpdp6d7qKhAAJs3umJZF5pmZ1Ro4z2a9gA== --osd-uuid 878937b3-bf51-45ae-8ada-423109d5e77a 2026-02-19T13:21:05.964 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:05.964+0000 7fe2abbf9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:05.965 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:05.965+0000 7fe2abbf9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:05.966 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:05.966+0000 7fe2abbf9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:05.966 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:05.966+0000 7fe2abbf9900 -1 bdev(0x5588fe946c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:21:05.967 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:05.967+0000 7fe2abbf9900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:21:06.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:21:06.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:21:06.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:21:06.396 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:21:06.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:21:06.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:21:06.696 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:21:06.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:06.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:21:06.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:21:06.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:21:06.710 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:06.710+0000 7f4a62e02900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:06.711 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:06.712+0000 7f4a62e02900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:06.712 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:06.712+0000 7f4a62e02900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:06.909 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:06.909+0000 7f4a62e02900 -1 Falling back to public interface 2026-02-19T13:21:06.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:21:06.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:06.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:21:06.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:06.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:06.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:06.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:06.922 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:06.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:06.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:21:07.141 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:07.315 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:07.315+0000 7f4a62e02900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:21:08.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:08.142 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:08.143 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:08.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:08.143 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:08.144 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:21:08.363 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/250882731,v1:127.0.0.1:6803/250882731] [v2:127.0.0.1:6804/250882731,v1:127.0.0.1:6805/250882731] exists,up 878937b3-bf51-45ae-8ada-423109d5e77a 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:21:08.364 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:08.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:21:08.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:21:08.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=b32f725b-6437-4af8-bdae-a7805d32e6c1 2026-02-19T13:21:08.367 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 b32f725b-6437-4af8-bdae-a7805d32e6c1 2026-02-19T13:21:08.367 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 b32f725b-6437-4af8-bdae-a7805d32e6c1' 2026-02-19T13:21:08.367 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:21:08.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBEDpdpDBSdFhAAWksHSjtn7IxRvWrkkyunPw== 2026-02-19T13:21:08.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBEDpdpDBSdFhAAWksHSjtn7IxRvWrkkyunPw=="}' 2026-02-19T13:21:08.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new b32f725b-6437-4af8-bdae-a7805d32e6c1 -i td/test-erasure-eio/1/new.json 2026-02-19T13:21:08.598 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:08.609 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:21:08.610 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBEDpdpDBSdFhAAWksHSjtn7IxRvWrkkyunPw== --osd-uuid b32f725b-6437-4af8-bdae-a7805d32e6c1 2026-02-19T13:21:08.626 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:08.626+0000 7f44c4404900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:08.627 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:08.628+0000 7f44c4404900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:08.628 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:08.629+0000 7f44c4404900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:08.629 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:08.629+0000 7f44c4404900 -1 bdev(0x55f0dcfecc00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:21:08.629 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:08.629+0000 7f44c4404900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:21:09.056 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:21:09.057 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:21:09.057 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:21:09.058 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:21:09.058 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:21:09.356 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:21:09.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:21:09.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:09.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:21:09.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:21:09.357 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:21:09.371 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:09.372+0000 7fb9ac908900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:09.373 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:09.373+0000 7fb9ac908900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:09.373 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:09.374+0000 7fb9ac908900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:09.543 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:09.543+0000 7fb9ac908900 -1 Falling back to public interface 2026-02-19T13:21:09.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:21:09.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:09.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:21:09.580 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:09.580 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:09.580 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:09.580 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:09.580 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:09.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:09.581 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:21:09.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:09.930 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:09.930+0000 7fb9ac908900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:21:10.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:10.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:10.800 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:10.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:10.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:10.801 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:21:11.024 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/612490885,v1:127.0.0.1:6811/612490885] [v2:127.0.0.1:6812/612490885,v1:127.0.0.1:6813/612490885] exists,up b32f725b-6437-4af8-bdae-a7805d32e6c1 2026-02-19T13:21:11.028 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:21:11.029 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:11.030 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:11.031 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:11.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:11.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:21:11.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:21:11.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:21:11.032 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:21:11.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=5f3cce6b-0200-4f44-a968-6479b00fe8f6 2026-02-19T13:21:11.032 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 5f3cce6b-0200-4f44-a968-6479b00fe8f6' 2026-02-19T13:21:11.032 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 5f3cce6b-0200-4f44-a968-6479b00fe8f6 2026-02-19T13:21:11.033 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:21:11.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBHDpdpvrtxAhAASBv95Vzd7JF4YF1bssRZUw== 2026-02-19T13:21:11.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBHDpdpvrtxAhAASBv95Vzd7JF4YF1bssRZUw=="}' 2026-02-19T13:21:11.040 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 5f3cce6b-0200-4f44-a968-6479b00fe8f6 -i td/test-erasure-eio/2/new.json 2026-02-19T13:21:11.256 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:21:11.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:21:11.268 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBHDpdpvrtxAhAASBv95Vzd7JF4YF1bssRZUw== --osd-uuid 5f3cce6b-0200-4f44-a968-6479b00fe8f6 2026-02-19T13:21:11.284 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:11.284+0000 7f0672ecd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:11.285 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:11.286+0000 7f0672ecd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:11.286 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:11.287+0000 7f0672ecd900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:11.287 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:11.287+0000 7f0672ecd900 -1 bdev(0x55d6464f4c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:21:11.287 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:11.287+0000 7f0672ecd900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:21:11.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:21:11.704 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:21:11.705 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:21:11.705 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:21:11.706 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:21:12.005 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:21:12.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:21:12.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:12.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:21:12.006 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:21:12.007 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:21:12.019 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:12.020+0000 7fd8636b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:12.021 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:12.021+0000 7fd8636b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:12.021 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:12.022+0000 7fd8636b0900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:12.216 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:12.216+0000 7fd8636b0900 -1 Falling back to public interface 2026-02-19T13:21:12.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:21:12.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:12.226 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:21:12.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:12.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:12.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:12.227 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:12.227 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:12.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:12.228 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:21:12.446 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:12.582 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:12.582+0000 7fd8636b0900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:21:13.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:13.448 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:13.449 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:13.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:13.449 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:13.450 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:21:13.675 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 18 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/3865176090,v1:127.0.0.1:6819/3865176090] [v2:127.0.0.1:6820/3865176090,v1:127.0.0.1:6821/3865176090] exists,up 5f3cce6b-0200-4f44-a968-6479b00fe8f6 2026-02-19T13:21:13.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:13.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:13.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:13.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:13.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:13.682 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:13.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:13.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:21:13.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:21:13.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:21:13.684 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:21:13.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=9f1fa315-31a1-4149-b20c-339c34104281 2026-02-19T13:21:13.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 9f1fa315-31a1-4149-b20c-339c34104281' 2026-02-19T13:21:13.684 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 9f1fa315-31a1-4149-b20c-339c34104281 2026-02-19T13:21:13.685 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:21:13.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBJDpdpXnE4KRAAgaM7zvKE1m4uEWJDcWrRsw== 2026-02-19T13:21:13.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBJDpdpXnE4KRAAgaM7zvKE1m4uEWJDcWrRsw=="}' 2026-02-19T13:21:13.691 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 9f1fa315-31a1-4149-b20c-339c34104281 -i td/test-erasure-eio/3/new.json 2026-02-19T13:21:13.909 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:21:13.921 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:21:13.922 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBJDpdpXnE4KRAAgaM7zvKE1m4uEWJDcWrRsw== --osd-uuid 9f1fa315-31a1-4149-b20c-339c34104281 2026-02-19T13:21:13.938 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:13.938+0000 7fb8aaff9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:13.939 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:13.939+0000 7fb8aaff9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:13.940 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:13.940+0000 7fb8aaff9900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:13.940 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:13.941+0000 7fb8aaff9900 -1 bdev(0x55bd5c6e2c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:21:13.941 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:13.941+0000 7fb8aaff9900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:21:14.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:21:14.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:21:14.383 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:21:14.383 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:21:14.384 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:21:14.707 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:21:14.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:21:14.708 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:14.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:21:14.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:21:14.709 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:21:14.722 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:14.722+0000 7f33502d2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:14.723 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:14.723+0000 7f33502d2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:14.724 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:14.724+0000 7f33502d2900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:14.928 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:21:14.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:14.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:21:14.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:14.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:14.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:14.929 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:14.929 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:14.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:14.930 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:21:14.936 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:14.936+0000 7f33502d2900 -1 Falling back to public interface 2026-02-19T13:21:15.148 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:15.246 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:15.246+0000 7f33502d2900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:21:16.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:16.149 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:16.150 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:16.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:16.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:16.151 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:21:16.372 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/738017179,v1:127.0.0.1:6827/738017179] [v2:127.0.0.1:6828/738017179,v1:127.0.0.1:6829/738017179] exists,up 9f1fa315-31a1-4149-b20c-339c34104281 2026-02-19T13:21:16.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:16.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:16.375 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:16.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:21:16.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:21:16.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:21:16.376 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:16.376 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:16.376 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:16.377 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:21:16.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:21:16.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:21:16.431 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:21:16.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:21:16.440 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:21:06.912+0000 7f4a62e02900 0 load: jerasure load: lrc 2026-02-19T13:21:16.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:355: TEST_rados_get_with_subreadall_eio_shard_0: local poolname=pool-jerasure 2026-02-19T13:21:16.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:356: TEST_rados_get_with_subreadall_eio_shard_0: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T13:21:16.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:21:16.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:21:16.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T13:21:16.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:21:16.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T13:21:16.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:21:16.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T13:21:16.720 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:21:16.721 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:21:17.083 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:21:17.094 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:21:18.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:21:18.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:21:18.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:21:18.096 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:21:18.096 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:21:18.097 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:21:18.097 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:21:18.098 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:21:18.098 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:21:18.098 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:21:18.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:21:18.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:21:18.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:21:18.180 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:21:18.181 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:21:18.181 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:21:18.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:21:18.407 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:18.407 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:21:18.407 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:21:18.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:21:18.407 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:18.407 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:21:18.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803781 2026-02-19T13:21:18.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803781 2026-02-19T13:21:18.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781' 2026-02-19T13:21:18.485 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:18.485 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:21:18.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-19T13:21:18.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-19T13:21:18.565 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852' 2026-02-19T13:21:18.566 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:18.566 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:21:18.642 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=77309411331 2026-02-19T13:21:18.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 77309411331 2026-02-19T13:21:18.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411331' 2026-02-19T13:21:18.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:18.643 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:21:18.721 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T13:21:18.721 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T13:21:18.721 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803781 1-55834574852 2-77309411331 3-111669149699' 2026-02-19T13:21:18.721 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:18.721 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803781 2026-02-19T13:21:18.721 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:18.722 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:21:18.722 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803781 2026-02-19T13:21:18.723 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:18.723 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803781 2026-02-19T13:21:18.724 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803781 2026-02-19T13:21:18.724 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803781' 2026-02-19T13:21:18.724 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:18.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803781 2026-02-19T13:21:18.944 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:21:19.945 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:21:19.945 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:20.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803781 2026-02-19T13:21:20.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:20.171 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-19T13:21:20.171 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:20.171 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:21:20.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574852 2026-02-19T13:21:20.172 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:20.173 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574852 2026-02-19T13:21:20.173 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574852 2026-02-19T13:21:20.174 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-19T13:21:20.174 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:21:20.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-19T13:21:20.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:20.393 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-77309411331 2026-02-19T13:21:20.393 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:20.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:21:20.395 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-77309411331 2026-02-19T13:21:20.395 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:20.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=77309411331 2026-02-19T13:21:20.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 77309411331' 2026-02-19T13:21:20.396 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 77309411331 2026-02-19T13:21:20.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:21:20.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 77309411332 -lt 77309411331 2026-02-19T13:21:20.620 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:20.621 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T13:21:20.621 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:20.621 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:21:20.622 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149699 2026-02-19T13:21:20.622 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:20.623 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149699 2026-02-19T13:21:20.623 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149699 2026-02-19T13:21:20.624 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T13:21:20.624 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:21:20.844 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-19T13:21:20.845 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:21:20.845 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:20.845 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:21.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:21:21.150 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:21:21.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:21:21.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:21:21.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:21:21.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:21:21.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:21:21.150 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:21:21.368 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:21:21.369 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:21:21.369 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:21.369 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:21.668 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:358: TEST_rados_get_with_subreadall_eio_shard_0: rados_put_get_data eio td/test-erasure-eio 0 recovery 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=0 2026-02-19T13:21:21.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-65583-0 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-65583-0 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-65583-0 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:21:21.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:21:21.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:21:21.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:21:21.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:21:21.671 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-65583-0 td/test-erasure-eio/ORIGINAL 2026-02-19T13:21:21.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-65583-0 td/test-erasure-eio 0 2026-02-19T13:21:21.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:21:21.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:21:21.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:21:21.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-65583-0 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=0 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-65583-0 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-0 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-0 2026-02-19T13:21:21.688 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:21:21.910 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:21:21.910 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:21.910 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:21:21.910 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:21:21.910 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T13:21:21.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:21:21.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=3 2026-02-19T13:21:21.911 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:21:21.911 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/3/type 2026-02-19T13:21:21.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:21:21.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 3 bluestore_debug_inject_read_err true 2026-02-19T13:21:21.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:21:21.913 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=3 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.3 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:21:21.914 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.3.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:21:21.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:21:21.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:21:21.980 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.3 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.3 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.3 ']' 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.3.asok 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:21:21.982 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.3.asok injectdataerr pool-jerasure obj-eio-65583-0 0 2026-02-19T13:21:22.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-65583-0 2026-02-19T13:21:22.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:21:22.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:21:22.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-65583-0 2026-02-19T13:21:22.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:21:22.046 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:21:22.047 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-65583-0 td/test-erasure-eio/COPY 2026-02-19T13:21:22.063 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:21:22.064 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:21:22.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-19T13:21:22.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-65583-0 2026-02-19T13:21:22.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:21:22.065 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-0 2026-02-19T13:21:22.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-0 2026-02-19T13:21:22.066 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:21:22.293 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:21:22.293 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:22.293 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:21:22.294 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:21:22.600 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:21:22.601 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-19T13:21:22.868 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already out. 2026-02-19T13:21:22.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-65583-0 2026-02-19T13:21:22.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:21:22.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-0 2026-02-19T13:21:22.881 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-19T13:21:22.882 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-0 2026-02-19T13:21:22.882 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:21:23.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:21:23.109 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:23.109 INFO:tasks.workunit.client.0.trial111.stderr:0' 2026-02-19T13:21:23.109 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-19T13:21:23.110 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-19T13:21:23.376 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already in. 2026-02-19T13:21:23.390 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:23.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:23.392 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:21:23.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:21:23.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:21:23.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:21:23.394 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:21:23.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:23.395 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:21:23.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:21:23.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:21:23.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:21:23.396 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:21:23.404 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:23.405+0000 7fba1bbdf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:23.405 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:23.407+0000 7fba1bbdf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:23.406 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:23.408+0000 7fba1bbdf900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:23.575 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:23.577+0000 7fba1bbdf900 -1 Falling back to public interface 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:23.616 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:21:23.840 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:23.969 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:23.970+0000 7fba1bbdf900 -1 osd.2 33 log_to_monitors true 2026-02-19T13:21:24.841 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:24.842 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:24.842 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:24.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:24.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:24.843 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:21:25.066 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 42 up_thru 19 down_at 34 last_clean_interval [18,33) [v2:127.0.0.1:6818/2717644150,v1:127.0.0.1:6819/2717644150] [v2:127.0.0.1:6820/2717644150,v1:127.0.0.1:6821/2717644150] exists,up 5f3cce6b-0200-4f44-a968-6479b00fe8f6 2026-02-19T13:21:25.067 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:21:25.068 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:21:25.069 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:21:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:21:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:21:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:21:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:21:25.146 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:21:25.146 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:21:25.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:21:25.366 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:25.366 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:21:25.366 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:21:25.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:21:25.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:25.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:21:25.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-19T13:21:25.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-19T13:21:25.443 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784' 2026-02-19T13:21:25.444 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:25.444 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:21:25.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-19T13:21:25.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-19T13:21:25.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574855' 2026-02-19T13:21:25.524 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:25.524 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:21:25.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=180388626435 2026-02-19T13:21:25.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 180388626435 2026-02-19T13:21:25.605 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-180388626435' 2026-02-19T13:21:25.606 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:25.606 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:21:25.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T13:21:25.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T13:21:25.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-180388626435 3-111669149702' 2026-02-19T13:21:25.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:25.684 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-19T13:21:25.684 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:25.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:21:25.685 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803784 2026-02-19T13:21:25.685 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:25.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803784 2026-02-19T13:21:25.686 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-19T13:21:25.686 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803784 2026-02-19T13:21:25.686 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-19T13:21:25.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:21:26.908 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:21:26.909 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:27.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-19T13:21:27.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:27.128 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-19T13:21:27.128 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:27.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:21:27.130 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574855 2026-02-19T13:21:27.130 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:27.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574855 2026-02-19T13:21:27.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-19T13:21:27.131 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574855 2026-02-19T13:21:27.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:21:27.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574856 -lt 55834574855 2026-02-19T13:21:27.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:27.353 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-180388626435 2026-02-19T13:21:27.353 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:27.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:21:27.354 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-180388626435 2026-02-19T13:21:27.354 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:27.355 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=180388626435 2026-02-19T13:21:27.355 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 180388626435 2026-02-19T13:21:27.356 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 180388626435' 2026-02-19T13:21:27.356 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:21:27.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 180388626435 -lt 180388626435 2026-02-19T13:21:27.576 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:27.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T13:21:27.576 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:27.577 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:21:27.578 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149702 2026-02-19T13:21:27.578 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:27.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149702 2026-02-19T13:21:27.579 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T13:21:27.579 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149702 2026-02-19T13:21:27.579 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:21:27.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149703 -lt 111669149702 2026-02-19T13:21:27.800 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:21:27.800 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:27.800 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:28.099 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:21:28.099 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:21:28.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:21:28.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:21:28.099 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:21:28.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:21:28.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:21:28.100 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:21:28.317 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:21:28.317 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:21:28.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:28.318 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:28.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:21:28.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:21:28.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:21:28.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:360: TEST_rados_get_with_subreadall_eio_shard_0: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:21:28.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:21:28.619 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:21:28.881 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:21:28.893 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:21:29.189 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:21:29.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:21:29.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:21:29.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:21:29.201 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:21:29.201 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:21:29.201 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:21:29.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:21:29.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:21:29.202 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:21:29.309 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:21:29.309 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:21:29.310 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:21:29.310 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:21:29.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:21:29.311 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:21:29.312 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:21:29.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:29.312 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:21:29.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:21:29.313 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:29.313 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:21:29.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:21:29.314 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:21:29.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:21:29.324 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:29.325 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:29.325 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:21:29.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:21:29.326 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:32: run: for func in $funcs 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:33: run: setup td/test-erasure-eio 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:122: setup: local dir=td/test-erasure-eio 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:123: setup: teardown td/test-erasure-eio 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:21:29.327 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:21:29.328 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:21:29.328 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:21:29.329 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:21:29.329 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:21:29.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:21:29.330 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:21:29.331 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:21:29.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:29.331 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:21:29.332 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:21:29.332 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:29.333 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:21:29.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:21:29.334 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:21:29.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:21:29.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:29.335 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:29.335 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:21:29.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:21:29.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:21:29.336 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:124: setup: mkdir -p td/test-erasure-eio 2026-02-19T13:21:29.337 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: get_asok_dir 2026-02-19T13:21:29.338 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:29.338 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:29.338 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:125: setup: mkdir -p /tmp/ceph-asok.65583 2026-02-19T13:21:29.339 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: ulimit -n 2026-02-19T13:21:29.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:126: setup: '[' 4096 -le 1024 ']' 2026-02-19T13:21:29.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:129: setup: '[' -z '' ']' 2026-02-19T13:21:29.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:130: setup: trap 'teardown td/test-erasure-eio 1' TERM HUP INT 2026-02-19T13:21:29.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:34: run: run_mon td/test-erasure-eio a 2026-02-19T13:21:29.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:439: run_mon: local dir=td/test-erasure-eio 2026-02-19T13:21:29.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:440: run_mon: shift 2026-02-19T13:21:29.340 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:441: run_mon: local id=a 2026-02-19T13:21:29.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:442: run_mon: shift 2026-02-19T13:21:29.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:443: run_mon: local data=td/test-erasure-eio/a 2026-02-19T13:21:29.341 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:446: run_mon: ceph-mon --id a --mkfs --mon-data=td/test-erasure-eio/a --run-dir=td/test-erasure-eio 2026-02-19T13:21:29.360 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: get_asok_path 2026-02-19T13:21:29.360 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:29.360 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:29.360 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:29.360 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:29.360 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:29.360 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:29.361 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:453: run_mon: ceph-mon --id a --osd-failsafe-full-ratio=.99 --mon-osd-full-ratio=.99 --mon-data-avail-crit=1 --mon-data-avail-warn=5 --paxos-propose-interval=0.1 --osd-crush-chooseleaf-type=0 --debug-mon 20 --debug-ms 20 --debug-paxos 20 --chdir= --mon-data=td/test-erasure-eio/a '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --mon-cluster-log-file=td/test-erasure-eio/log --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mon-allow-pool-delete --mon-allow-pool-size-one --osd-pool-default-pg-autoscale-mode off --mon-osd-backfillfull-ratio .99 --mon-warn-on-insecure-global-id-reclaim-allowed=false 2026-02-19T13:21:29.382 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: cat 2026-02-19T13:21:29.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a fsid 2026-02-19T13:21:29.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:21:29.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:21:29.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=fsid 2026-02-19T13:21:29.383 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .fsid 2026-02-19T13:21:29.385 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:21:29.385 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:21:29.385 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:21:29.385 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:29.385 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:29.385 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:29.385 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:21:29.385 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:21:29.386 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get fsid 2026-02-19T13:21:29.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:478: run_mon: get_config mon a mon_host 2026-02-19T13:21:29.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1116: get_config: local daemon=mon 2026-02-19T13:21:29.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1117: get_config: local id=a 2026-02-19T13:21:29.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1118: get_config: local config=mon_host 2026-02-19T13:21:29.445 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1123: get_config: jq -r .mon_host 2026-02-19T13:21:29.446 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: get_asok_path mon.a 2026-02-19T13:21:29.447 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:21:29.447 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:21:29.447 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:29.447 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:29.447 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:29.447 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:21:29.447 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: CEPH_ARGS= 2026-02-19T13:21:29.447 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1120: get_config: ceph --format json daemon /tmp/ceph-asok.65583/ceph-mon.a.asok config get mon_host 2026-02-19T13:21:29.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:35: run: run_mgr td/test-erasure-eio x 2026-02-19T13:21:29.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:544: run_mgr: local dir=td/test-erasure-eio 2026-02-19T13:21:29.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:545: run_mgr: shift 2026-02-19T13:21:29.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:546: run_mgr: local id=x 2026-02-19T13:21:29.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:547: run_mgr: shift 2026-02-19T13:21:29.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:548: run_mgr: local data=td/test-erasure-eio/x 2026-02-19T13:21:29.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:550: run_mgr: ceph config set mgr mgr_pool false --force 2026-02-19T13:21:29.616 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: get_asok_path 2026-02-19T13:21:29.617 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:29.617 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:29.617 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:29.617 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:29.617 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:29.617 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:29.617 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: realpath /home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:21:29.618 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:552: run_mgr: ceph-mgr --id x --osd-failsafe-full-ratio=.99 --debug-mgr 20 --debug-objecter 20 --debug-ms 20 --debug-paxos 20 --chdir= --mgr-data=td/test-erasure-eio/x '--log-file=td/test-erasure-eio/$name.log' '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --run-dir=td/test-erasure-eio '--pid-file=td/test-erasure-eio/$name.pid' --mgr-module-path=/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr 2026-02-19T13:21:29.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:36: run: create_pool rbd 4 2026-02-19T13:21:29.635 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create rbd 4 2026-02-19T13:21:29.737 INFO:tasks.workunit.client.0.trial111.stderr:pool 'rbd' already exists 2026-02-19T13:21:29.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:21:30.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: get_asok_path mon.a 2026-02-19T13:21:30.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=mon.a 2026-02-19T13:21:30.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n mon.a ']' 2026-02-19T13:21:30.750 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:30.750 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:30.750 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:30.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-mon.a.asok 2026-02-19T13:21:30.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: CEPH_ARGS= 2026-02-19T13:21:30.751 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:39: run: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-mon.a.asok log flush 2026-02-19T13:21:30.805 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:21:30.812 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:40: run: grep 'load: jerasure.*lrc' td/test-erasure-eio/mon.a.log 2026-02-19T13:21:30.814 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:21:29.376+0000 7f12d098ddc0 0 load: jerasure load: lrc 2026-02-19T13:21:30.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:41: run: TEST_rados_get_with_subreadall_eio_shard_1 td/test-erasure-eio 2026-02-19T13:21:30.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:364: TEST_rados_get_with_subreadall_eio_shard_1: local dir=td/test-erasure-eio 2026-02-19T13:21:30.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:365: TEST_rados_get_with_subreadall_eio_shard_1: local shard_id=1 2026-02-19T13:21:30.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:367: TEST_rados_get_with_subreadall_eio_shard_1: setup_osds 4 2026-02-19T13:21:30.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:47: setup_osds: local count=4 2026-02-19T13:21:30.814 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:48: setup_osds: shift 2026-02-19T13:21:30.815 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: expr 4 - 1 2026-02-19T13:21:30.816 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: seq 0 3 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 0 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=0 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/0 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/0' 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/0/journal' 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:21:30.817 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:21:30.818 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/0 2026-02-19T13:21:30.819 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:21:30.820 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=44625538-54ff-40f6-8f5b-def40c6dbc4b 2026-02-19T13:21:30.820 INFO:tasks.workunit.client.0.trial111.stdout:add osd0 44625538-54ff-40f6-8f5b-def40c6dbc4b 2026-02-19T13:21:30.820 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd0 44625538-54ff-40f6-8f5b-def40c6dbc4b' 2026-02-19T13:21:30.820 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:21:30.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBaDpdpggClMRAAszWsOelsBhsD+Z4bTZc3Aw== 2026-02-19T13:21:30.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBaDpdpggClMRAAszWsOelsBhsD+Z4bTZc3Aw=="}' 2026-02-19T13:21:30.832 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 44625538-54ff-40f6-8f5b-def40c6dbc4b -i td/test-erasure-eio/0/new.json 2026-02-19T13:21:31.052 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:31.064 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/0/new.json 2026-02-19T13:21:31.065 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBaDpdpggClMRAAszWsOelsBhsD+Z4bTZc3Aw== --osd-uuid 44625538-54ff-40f6-8f5b-def40c6dbc4b 2026-02-19T13:21:31.081 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:31.081+0000 7f06c66c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:31.082 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:31.082+0000 7f06c66c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:31.083 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:31.083+0000 7f06c66c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:31.083 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:31.083+0000 7f06c66c6900 -1 bdev(0x560ea3b18c00 td/test-erasure-eio/0/block) open stat got: (1) Operation not permitted 2026-02-19T13:21:31.083 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:31.083+0000 7f06c66c6900 -1 bluestore(td/test-erasure-eio/0) _read_fsid unparsable uuid 2026-02-19T13:21:31.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/0/keyring 2026-02-19T13:21:31.528 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:21:31.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd0 key to auth repository 2026-02-19T13:21:31.529 INFO:tasks.workunit.client.0.trial111.stdout:adding osd0 key to auth repository 2026-02-19T13:21:31.529 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/0/keyring auth add osd.0 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:21:31.828 INFO:tasks.workunit.client.0.trial111.stdout:start osd.0 2026-02-19T13:21:31.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.0 2026-02-19T13:21:31.829 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 0 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/0 --osd-journal=td/test-erasure-eio/0/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:31.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:21:31.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:21:31.830 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:21:31.843 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:31.843+0000 7f445dd0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:31.844 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:31.845+0000 7f445dd0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:31.845 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:31.846+0000 7f445dd0e900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 0 2026-02-19T13:21:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=0 2026-02-19T13:21:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:32.053 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:32.054 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:32.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:32.054 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:21:32.069 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:32.070+0000 7f445dd0e900 -1 Falling back to public interface 2026-02-19T13:21:32.273 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:32.425 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:32.425+0000 7f445dd0e900 -1 osd.0 0 log_to_monitors true 2026-02-19T13:21:33.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:33.274 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:33.275 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:33.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:33.275 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:33.276 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.0 up' 2026-02-19T13:21:33.496 INFO:tasks.workunit.client.0.trial111.stdout:osd.0 up in weight 1 up_from 6 up_thru 6 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6802/3125992736,v1:127.0.0.1:6803/3125992736] [v2:127.0.0.1:6804/3125992736,v1:127.0.0.1:6805/3125992736] exists,up 44625538-54ff-40f6-8f5b-def40c6dbc4b 2026-02-19T13:21:33.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:33.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:33.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:33.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:21:33.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 1 2026-02-19T13:21:33.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:33.500 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=1 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/1 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/1' 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/1/journal' 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:21:33.501 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:21:33.502 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/1 2026-02-19T13:21:33.503 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:21:33.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=36fb13e9-cb7b-4e53-a5c7-dc0bdaaf1034 2026-02-19T13:21:33.504 INFO:tasks.workunit.client.0.trial111.stdout:add osd1 36fb13e9-cb7b-4e53-a5c7-dc0bdaaf1034 2026-02-19T13:21:33.504 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd1 36fb13e9-cb7b-4e53-a5c7-dc0bdaaf1034' 2026-02-19T13:21:33.505 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:21:33.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBdDpdpbvyIHhAAa9jhtKVRwKSdaMR0u/hYGQ== 2026-02-19T13:21:33.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBdDpdpbvyIHhAAa9jhtKVRwKSdaMR0u/hYGQ=="}' 2026-02-19T13:21:33.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 36fb13e9-cb7b-4e53-a5c7-dc0bdaaf1034 -i td/test-erasure-eio/1/new.json 2026-02-19T13:21:33.732 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:33.744 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/1/new.json 2026-02-19T13:21:33.745 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBdDpdpbvyIHhAAa9jhtKVRwKSdaMR0u/hYGQ== --osd-uuid 36fb13e9-cb7b-4e53-a5c7-dc0bdaaf1034 2026-02-19T13:21:33.760 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:33.761+0000 7f2096651900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:33.761 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:33.762+0000 7f2096651900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:33.762 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:33.763+0000 7f2096651900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:33.762 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:33.763+0000 7f2096651900 -1 bdev(0x564ca8bd8c00 td/test-erasure-eio/1/block) open stat got: (1) Operation not permitted 2026-02-19T13:21:33.763 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:33.763+0000 7f2096651900 -1 bluestore(td/test-erasure-eio/1) _read_fsid unparsable uuid 2026-02-19T13:21:34.160 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/1/keyring 2026-02-19T13:21:34.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:21:34.161 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd1 key to auth repository 2026-02-19T13:21:34.162 INFO:tasks.workunit.client.0.trial111.stdout:adding osd1 key to auth repository 2026-02-19T13:21:34.162 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/1/keyring auth add osd.1 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:21:34.459 INFO:tasks.workunit.client.0.trial111.stdout:start osd.1 2026-02-19T13:21:34.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.1 2026-02-19T13:21:34.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 1 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/1 --osd-journal=td/test-erasure-eio/1/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:34.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:21:34.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:21:34.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:21:34.474 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:34.474+0000 7f2e1a1b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:34.475 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:34.476+0000 7f2e1a1b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:34.476 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:34.477+0000 7f2e1a1b6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:34.680 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 1 2026-02-19T13:21:34.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:34.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=1 2026-02-19T13:21:34.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:34.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:34.681 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:34.681 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:34.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:34.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:34.682 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:21:34.683 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:34.683+0000 7f2e1a1b6900 -1 Falling back to public interface 2026-02-19T13:21:34.900 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:35.035 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:35.035+0000 7f2e1a1b6900 -1 osd.1 0 log_to_monitors true 2026-02-19T13:21:35.901 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:35.902 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:35.902 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:35.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:35.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:35.903 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.1 up' 2026-02-19T13:21:36.124 INFO:tasks.workunit.client.0.trial111.stdout:osd.1 up in weight 1 up_from 13 up_thru 13 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6810/181075967,v1:127.0.0.1:6811/181075967] [v2:127.0.0.1:6812/181075967,v1:127.0.0.1:6813/181075967] exists,up 36fb13e9-cb7b-4e53-a5c7-dc0bdaaf1034 2026-02-19T13:21:36.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:36.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:36.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:36.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:21:36.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 2 2026-02-19T13:21:36.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:36.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:21:36.128 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=2 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:36.129 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:36.130 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=6e401355-bfd0-4449-815d-9cf91494b962 2026-02-19T13:21:36.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd2 6e401355-bfd0-4449-815d-9cf91494b962' 2026-02-19T13:21:36.132 INFO:tasks.workunit.client.0.trial111.stdout:add osd2 6e401355-bfd0-4449-815d-9cf91494b962 2026-02-19T13:21:36.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:21:36.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBgDpdpmE9QCBAA8XDB6j3UcY7kNsUO6ule2g== 2026-02-19T13:21:36.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBgDpdpmE9QCBAA8XDB6j3UcY7kNsUO6ule2g=="}' 2026-02-19T13:21:36.139 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 6e401355-bfd0-4449-815d-9cf91494b962 -i td/test-erasure-eio/2/new.json 2026-02-19T13:21:36.360 INFO:tasks.workunit.client.0.trial111.stdout:2 2026-02-19T13:21:36.371 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/2/new.json 2026-02-19T13:21:36.372 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBgDpdpmE9QCBAA8XDB6j3UcY7kNsUO6ule2g== --osd-uuid 6e401355-bfd0-4449-815d-9cf91494b962 2026-02-19T13:21:36.388 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:36.388+0000 7f395c0c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:36.389 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:36.389+0000 7f395c0c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:36.390 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:36.390+0000 7f395c0c6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:36.391 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:36.391+0000 7f395c0c6900 -1 bdev(0x55d436fa2c00 td/test-erasure-eio/2/block) open stat got: (1) Operation not permitted 2026-02-19T13:21:36.391 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:36.391+0000 7f395c0c6900 -1 bluestore(td/test-erasure-eio/2) _read_fsid unparsable uuid 2026-02-19T13:21:36.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/2/keyring 2026-02-19T13:21:36.825 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:21:36.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd2 key to auth repository 2026-02-19T13:21:36.826 INFO:tasks.workunit.client.0.trial111.stdout:adding osd2 key to auth repository 2026-02-19T13:21:36.826 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/2/keyring auth add osd.2 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:21:37.126 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:21:37.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.2 2026-02-19T13:21:37.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:37.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:21:37.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:21:37.127 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:21:37.141 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:37.141+0000 7ff6a0378900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:37.142 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:37.142+0000 7ff6a0378900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:37.143 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:37.143+0000 7ff6a0378900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:37.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 2 2026-02-19T13:21:37.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:37.352 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:21:37.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:37.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:37.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:37.353 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:37.353 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:37.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:37.354 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:21:37.355 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:37.355+0000 7ff6a0378900 -1 Falling back to public interface 2026-02-19T13:21:37.570 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:37.701 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:37.702+0000 7ff6a0378900 -1 osd.2 0 log_to_monitors true 2026-02-19T13:21:38.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:38.572 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:38.573 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:38.573 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:38.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:38.574 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:21:38.792 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 19 up_thru 19 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6818/1032968075,v1:127.0.0.1:6819/1032968075] [v2:127.0.0.1:6820/1032968075,v1:127.0.0.1:6821/1032968075] exists,up 6e401355-bfd0-4449-815d-9cf91494b962 2026-02-19T13:21:38.795 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:50: setup_osds: for id in $(seq 0 $(expr $count - 1)) 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:51: setup_osds: run_osd td/test-erasure-eio 3 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:624: run_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:625: run_osd: shift 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:626: run_osd: local id=3 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:627: run_osd: shift 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:628: run_osd: local osd_data=td/test-erasure-eio/3 2026-02-19T13:21:38.796 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:630: run_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:631: run_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:632: run_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:633: run_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:634: run_osd: ceph_args+=' --osd-data=td/test-erasure-eio/3' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:635: run_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/3/journal' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:636: run_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:637: run_osd: ceph_args+= 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:638: run_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: get_asok_path 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:38.797 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:639: run_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:640: run_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:641: run_osd: ceph_args+=' --debug-ms=1' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:642: run_osd: ceph_args+=' --debug-monc=20' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:643: run_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:644: run_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:645: run_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:646: run_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:647: run_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:648: run_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:38.798 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:649: run_osd: ceph_args+=' ' 2026-02-19T13:21:38.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:650: run_osd: ceph_args+= 2026-02-19T13:21:38.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:651: run_osd: mkdir -p td/test-erasure-eio/3 2026-02-19T13:21:38.799 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: uuidgen 2026-02-19T13:21:38.799 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:653: run_osd: local uuid=391a1e97-3b52-49bb-ab24-d0266d0f3b47 2026-02-19T13:21:38.799 INFO:tasks.workunit.client.0.trial111.stdout:add osd3 391a1e97-3b52-49bb-ab24-d0266d0f3b47 2026-02-19T13:21:38.800 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:654: run_osd: echo 'add osd3 391a1e97-3b52-49bb-ab24-d0266d0f3b47' 2026-02-19T13:21:38.800 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: ceph-authtool --gen-print-key 2026-02-19T13:21:38.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:655: run_osd: OSD_SECRET=AQBiDpdpyuAjMBAAgsy/md1Z3+122FU3H5ZgvA== 2026-02-19T13:21:38.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:656: run_osd: echo '{"cephx_secret": "AQBiDpdpyuAjMBAAgsy/md1Z3+122FU3H5ZgvA=="}' 2026-02-19T13:21:38.807 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:657: run_osd: ceph osd new 391a1e97-3b52-49bb-ab24-d0266d0f3b47 -i td/test-erasure-eio/3/new.json 2026-02-19T13:21:39.025 INFO:tasks.workunit.client.0.trial111.stdout:3 2026-02-19T13:21:39.036 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:658: run_osd: rm td/test-erasure-eio/3/new.json 2026-02-19T13:21:39.037 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:659: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops --mkfs --key AQBiDpdpyuAjMBAAgsy/md1Z3+122FU3H5ZgvA== --osd-uuid 391a1e97-3b52-49bb-ab24-d0266d0f3b47 2026-02-19T13:21:39.053 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:39.053+0000 7f401ddab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:39.054 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:39.054+0000 7f401ddab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:39.055 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:39.055+0000 7f401ddab900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:39.055 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:39.055+0000 7f401ddab900 -1 bdev(0x557e10f54c00 td/test-erasure-eio/3/block) open stat got: (1) Operation not permitted 2026-02-19T13:21:39.055 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:39.055+0000 7f401ddab900 -1 bluestore(td/test-erasure-eio/3) _read_fsid unparsable uuid 2026-02-19T13:21:39.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:661: run_osd: local key_fn=td/test-erasure-eio/3/keyring 2026-02-19T13:21:39.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:662: run_osd: cat 2026-02-19T13:21:39.475 INFO:tasks.workunit.client.0.trial111.stdout:adding osd3 key to auth repository 2026-02-19T13:21:39.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:666: run_osd: echo adding osd3 key to auth repository 2026-02-19T13:21:39.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:667: run_osd: ceph -i td/test-erasure-eio/3/keyring auth add osd.3 osd 'allow *' mon 'allow profile osd' mgr 'allow profile osd' 2026-02-19T13:21:39.778 INFO:tasks.workunit.client.0.trial111.stdout:start osd.3 2026-02-19T13:21:39.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:668: run_osd: echo start osd.3 2026-02-19T13:21:39.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:669: run_osd: ceph-osd -i 3 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/3 --osd-journal=td/test-erasure-eio/3/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 --debug-ms=1 --debug-monc=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:39.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: ceph osd dump --format=json 2026-02-19T13:21:39.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: jq '.flags_set[]' 2026-02-19T13:21:39.779 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:672: run_osd: grep -q '"noup"' 2026-02-19T13:21:39.793 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:39.793+0000 7fc0e5ec6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:39.794 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:39.794+0000 7fc0e5ec6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:39.795 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:39.795+0000 7fc0e5ec6900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:40.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:675: run_osd: wait_for_osd up 3 2026-02-19T13:21:40.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:40.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=3 2026-02-19T13:21:40.000 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:40.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:40.001 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:40.001 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:40.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:40.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:40.002 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:21:40.002 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:40.001+0000 7fc0e5ec6900 -1 Falling back to public interface 2026-02-19T13:21:40.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:40.365 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:40.366+0000 7fc0e5ec6900 -1 osd.3 0 log_to_monitors true 2026-02-19T13:21:41.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:41.220 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:41.220 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:41.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:41.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:41.221 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.3 up' 2026-02-19T13:21:41.439 INFO:tasks.workunit.client.0.trial111.stdout:osd.3 up in weight 1 up_from 26 up_thru 27 down_at 0 last_clean_interval [0,0) [v2:127.0.0.1:6826/3013286035,v1:127.0.0.1:6827/3013286035] [v2:127.0.0.1:6828/3013286035,v1:127.0.0.1:6829/3013286035] exists,up 391a1e97-3b52-49bb-ab24-d0266d0f3b47 2026-02-19T13:21:41.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:41.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:41.441 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: get_asok_path osd.0 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.0 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.0 ']' 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.0.asok 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: CEPH_ARGS= 2026-02-19T13:21:41.442 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:55: setup_osds: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.0.asok log flush 2026-02-19T13:21:41.500 INFO:tasks.workunit.client.0.trial111.stdout:{} 2026-02-19T13:21:41.507 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:56: setup_osds: grep 'load: jerasure.*lrc' td/test-erasure-eio/osd.0.log 2026-02-19T13:21:41.509 INFO:tasks.workunit.client.0.trial111.stdout:2026-02-19T13:21:32.073+0000 7f445dd0e900 0 load: jerasure load: lrc 2026-02-19T13:21:41.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:369: TEST_rados_get_with_subreadall_eio_shard_1: local poolname=pool-jerasure 2026-02-19T13:21:41.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:370: TEST_rados_get_with_subreadall_eio_shard_1: create_erasure_coded_pool pool-jerasure 2 1 2026-02-19T13:21:41.510 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:67: create_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:21:41.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:68: create_erasure_coded_pool: shift 2026-02-19T13:21:41.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:69: create_erasure_coded_pool: local k=2 2026-02-19T13:21:41.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:70: create_erasure_coded_pool: shift 2026-02-19T13:21:41.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:71: create_erasure_coded_pool: local m=1 2026-02-19T13:21:41.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:72: create_erasure_coded_pool: shift 2026-02-19T13:21:41.511 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:74: create_erasure_coded_pool: ceph osd erasure-code-profile set myprofile plugin=jerasure k=2 m=1 crush-failure-domain=osd 2026-02-19T13:21:41.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:78: create_erasure_coded_pool: create_pool pool-jerasure 1 1 erasure myprofile 2026-02-19T13:21:41.789 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:532: create_pool: ceph osd pool create pool-jerasure 1 1 erasure myprofile 2026-02-19T13:21:42.154 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' already exists 2026-02-19T13:21:42.167 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:533: create_pool: sleep 1 2026-02-19T13:21:43.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:80: create_erasure_coded_pool: wait_for_clean 2026-02-19T13:21:43.168 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:21:43.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:21:43.169 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:21:43.169 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:21:43.170 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:21:43.170 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:21:43.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:21:43.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:21:43.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:21:43.250 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:21:43.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:21:43.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:21:43.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:21:43.251 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:21:43.251 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:21:43.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:21:43.471 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:43.471 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:21:43.471 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:21:43.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:21:43.471 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:43.472 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:21:43.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803780 2026-02-19T13:21:43.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803780 2026-02-19T13:21:43.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780' 2026-02-19T13:21:43.551 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:43.551 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:21:43.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574852 2026-02-19T13:21:43.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574852 2026-02-19T13:21:43.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574852' 2026-02-19T13:21:43.631 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:43.631 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:21:43.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=81604378627 2026-02-19T13:21:43.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 81604378627 2026-02-19T13:21:43.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-81604378627' 2026-02-19T13:21:43.712 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:43.712 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:21:43.790 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149699 2026-02-19T13:21:43.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149699 2026-02-19T13:21:43.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803780 1-55834574852 2-81604378627 3-111669149699' 2026-02-19T13:21:43.791 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:43.791 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803780 2026-02-19T13:21:43.791 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:43.792 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:21:43.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803780 2026-02-19T13:21:43.793 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:43.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803780 2026-02-19T13:21:43.794 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803780' 2026-02-19T13:21:43.794 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803780 2026-02-19T13:21:43.794 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:44.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803778 -lt 25769803780 2026-02-19T13:21:44.021 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:21:45.022 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:21:45.023 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:45.245 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803781 -lt 25769803780 2026-02-19T13:21:45.246 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:45.246 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574852 2026-02-19T13:21:45.246 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:45.247 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:21:45.247 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574852 2026-02-19T13:21:45.248 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:45.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574852 2026-02-19T13:21:45.249 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574852' 2026-02-19T13:21:45.249 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574852 2026-02-19T13:21:45.250 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:21:45.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574852 -lt 55834574852 2026-02-19T13:21:45.474 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:45.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-81604378627 2026-02-19T13:21:45.475 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:45.475 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:21:45.476 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-81604378627 2026-02-19T13:21:45.476 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:45.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=81604378627 2026-02-19T13:21:45.477 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 81604378627' 2026-02-19T13:21:45.478 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 81604378627 2026-02-19T13:21:45.478 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:21:45.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 81604378628 -lt 81604378627 2026-02-19T13:21:45.696 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:45.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149699 2026-02-19T13:21:45.697 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:45.697 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:21:45.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149699 2026-02-19T13:21:45.698 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:45.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149699 2026-02-19T13:21:45.699 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149699' 2026-02-19T13:21:45.700 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149699 2026-02-19T13:21:45.700 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:21:45.920 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149699 -lt 111669149699 2026-02-19T13:21:45.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:21:45.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:45.921 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:46.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:21:46.219 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:21:46.219 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:21:46.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:21:46.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:21:46.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:21:46.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:21:46.220 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:21:46.438 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:21:46.438 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:21:46.439 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:46.439 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:46.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:21:46.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:21:46.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:21:46.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:372: TEST_rados_get_with_subreadall_eio_shard_1: rados_put_get_data eio td/test-erasure-eio 1 recovery 2026-02-19T13:21:46.738 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:147: rados_put_get_data: local inject=eio 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:148: rados_put_get_data: shift 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:149: rados_put_get_data: local dir=td/test-erasure-eio 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:150: rados_put_get_data: shift 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:151: rados_put_get_data: local shard_id=1 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:152: rados_put_get_data: shift 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:153: rados_put_get_data: local arg=recovery 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:157: rados_put_get_data: local poolname=pool-jerasure 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:158: rados_put_get_data: local objname=obj-eio-65583-1 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:159: rados_put_get_data: rados_put td/test-erasure-eio pool-jerasure obj-eio-65583-1 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:90: rados_put: local dir=td/test-erasure-eio 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:91: rados_put: local poolname=pool-jerasure 2026-02-19T13:21:46.739 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:92: rados_put: local objname=obj-eio-65583-1 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 AAA 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 BBB 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 CCCC 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:94: rados_put: for marker in AAA BBB CCCC DDDD 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:95: rados_put: printf '%*s' 1024 DDDD 2026-02-19T13:21:46.740 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:100: rados_put: rados --pool pool-jerasure put obj-eio-65583-1 td/test-erasure-eio/ORIGINAL 2026-02-19T13:21:46.756 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:160: rados_put_get_data: inject_eio ec data pool-jerasure obj-eio-65583-1 td/test-erasure-eio 1 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2481: inject_eio: local pooltype=ec 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2482: inject_eio: shift 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2483: inject_eio: local which=data 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2484: inject_eio: shift 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2485: inject_eio: local poolname=pool-jerasure 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2486: inject_eio: shift 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2487: inject_eio: local objname=obj-eio-65583-1 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2488: inject_eio: shift 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2489: inject_eio: local dir=td/test-erasure-eio 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2490: inject_eio: shift 2026-02-19T13:21:46.757 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2491: inject_eio: local shard_id=1 2026-02-19T13:21:46.758 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2492: inject_eio: shift 2026-02-19T13:21:46.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: get_osds pool-jerasure obj-eio-65583-1 2026-02-19T13:21:46.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:21:46.758 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-1 2026-02-19T13:21:46.758 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-1 2026-02-19T13:21:46.758 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:21:46.982 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:21:46.982 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:46.982 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:21:46.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:21:46.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: initial_osds=('3' '1' '2') 2026-02-19T13:21:46.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2494: inject_eio: local -a initial_osds 2026-02-19T13:21:46.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2495: inject_eio: local osd_id=1 2026-02-19T13:21:46.983 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2496: inject_eio: '[' ec '!=' ec ']' 2026-02-19T13:21:46.983 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: cat td/test-erasure-eio/1/type 2026-02-19T13:21:46.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2499: inject_eio: type=bluestore 2026-02-19T13:21:46.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2500: inject_eio: set_config osd 1 bluestore_debug_inject_read_err true 2026-02-19T13:21:46.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1152: set_config: local daemon=osd 2026-02-19T13:21:46.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1153: set_config: local id=1 2026-02-19T13:21:46.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1154: set_config: local config=bluestore_debug_inject_read_err 2026-02-19T13:21:46.985 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1155: set_config: local value=true 2026-02-19T13:21:46.985 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: jq 'has("success")' 2026-02-19T13:21:46.986 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: get_asok_path osd.1 2026-02-19T13:21:46.986 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:21:46.986 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:21:46.986 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:46.986 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:46.986 INFO:tasks.workunit.client.0.trial111.stderr:////home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:46.986 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:21:46.986 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: env CEPH_ARGS= ceph --format json daemon /tmp/ceph-asok.65583/ceph-osd.1.asok config set bluestore_debug_inject_read_err true 2026-02-19T13:21:47.049 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1159: set_config: test true == true 2026-02-19T13:21:47.049 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2501: inject_eio: local loop=0 2026-02-19T13:21:47.050 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2503: inject_eio: grep -q Invalid 2026-02-19T13:21:47.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: get_asok_path osd.1 2026-02-19T13:21:47.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name=osd.1 2026-02-19T13:21:47.050 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n osd.1 ']' 2026-02-19T13:21:47.051 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: get_asok_dir 2026-02-19T13:21:47.051 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:47.052 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:47.052 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:109: get_asok_path: echo /tmp/ceph-asok.65583/ceph-osd.1.asok 2026-02-19T13:21:47.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: CEPH_ARGS= 2026-02-19T13:21:47.052 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2502: inject_eio: ceph --admin-daemon /tmp/ceph-asok.65583/ceph-osd.1.asok injectdataerr pool-jerasure obj-eio-65583-1 1 2026-02-19T13:21:47.117 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:161: rados_put_get_data: rados_get td/test-erasure-eio pool-jerasure obj-eio-65583-1 2026-02-19T13:21:47.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:104: rados_get: local dir=td/test-erasure-eio 2026-02-19T13:21:47.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:105: rados_get: local poolname=pool-jerasure 2026-02-19T13:21:47.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:106: rados_get: local objname=obj-eio-65583-1 2026-02-19T13:21:47.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:107: rados_get: local expect=ok 2026-02-19T13:21:47.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:112: rados_get: '[' ok = fail ']' 2026-02-19T13:21:47.118 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:120: rados_get: rados --pool pool-jerasure get obj-eio-65583-1 td/test-erasure-eio/COPY 2026-02-19T13:21:47.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:121: rados_get: diff td/test-erasure-eio/ORIGINAL td/test-erasure-eio/COPY 2026-02-19T13:21:47.135 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:122: rados_get: rm td/test-erasure-eio/COPY 2026-02-19T13:21:47.136 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:163: rados_put_get_data: '[' recovery = recovery ']' 2026-02-19T13:21:47.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: get_osds pool-jerasure obj-eio-65583-1 2026-02-19T13:21:47.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:21:47.137 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-1 2026-02-19T13:21:47.138 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-1 2026-02-19T13:21:47.138 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:21:47.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:21:47.365 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:47.365 INFO:tasks.workunit.client.0.trial111.stderr:2' 2026-02-19T13:21:47.365 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 2 2026-02-19T13:21:47.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: initial_osds=('3' '1' '2') 2026-02-19T13:21:47.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:170: rados_put_get_data: local -a initial_osds 2026-02-19T13:21:47.365 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:171: rados_put_get_data: local last_osd=2 2026-02-19T13:21:47.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:173: rados_put_get_data: kill_daemons td/test-erasure-eio TERM osd.2 2026-02-19T13:21:47.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:21:47.366 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:21:47.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:21:47.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:21:47.366 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:21:47.672 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:21:47.672 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:174: rados_put_get_data: ceph osd out 2 2026-02-19T13:21:47.942 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already out. 2026-02-19T13:21:47.955 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: get_osds pool-jerasure obj-eio-65583-1 2026-02-19T13:21:47.955 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:175: rados_put_get_data: grep '\<2\>' 2026-02-19T13:21:47.955 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1012: get_osds: local poolname=pool-jerasure 2026-02-19T13:21:47.955 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1013: get_osds: local objectname=obj-eio-65583-1 2026-02-19T13:21:47.956 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: ceph --format json osd map pool-jerasure obj-eio-65583-1 2026-02-19T13:21:47.957 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: jq '.acting | .[]' 2026-02-19T13:21:48.177 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1016: get_osds: local 'osds=3 2026-02-19T13:21:48.178 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:48.178 INFO:tasks.workunit.client.0.trial111.stderr:0' 2026-02-19T13:21:48.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1018: get_osds: echo 3 1 0 2026-02-19T13:21:48.178 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:176: rados_put_get_data: ceph osd in 2 2026-02-19T13:21:48.446 INFO:tasks.workunit.client.0.trial111.stderr:osd.2 is already in. 2026-02-19T13:21:48.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:177: rados_put_get_data: activate_osd td/test-erasure-eio 2 2026-02-19T13:21:48.460 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:837: activate_osd: local dir=td/test-erasure-eio 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:838: activate_osd: shift 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:839: activate_osd: local id=2 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:840: activate_osd: shift 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:841: activate_osd: local osd_data=td/test-erasure-eio/2 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:843: activate_osd: local 'ceph_args=--fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true ' 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:844: activate_osd: ceph_args+=' --osd-failsafe-full-ratio=.99' 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:845: activate_osd: ceph_args+=' --osd-journal-size=100' 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:846: activate_osd: ceph_args+=' --osd-scrub-load-threshold=2000' 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:847: activate_osd: ceph_args+=' --osd-data=td/test-erasure-eio/2' 2026-02-19T13:21:48.461 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:848: activate_osd: ceph_args+=' --osd-journal=td/test-erasure-eio/2/journal' 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:849: activate_osd: ceph_args+=' --chdir=' 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:850: activate_osd: ceph_args+= 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:851: activate_osd: ceph_args+=' --run-dir=td/test-erasure-eio' 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: get_asok_path 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:107: get_asok_path: local name= 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:108: get_asok_path: '[' -n '' ']' 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: get_asok_dir 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:111: get_asok_path: echo '/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:852: activate_osd: ceph_args+=' --admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:853: activate_osd: ceph_args+=' --debug-osd=20' 2026-02-19T13:21:48.462 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:854: activate_osd: ceph_args+=' --log-file=td/test-erasure-eio/$name.log' 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:855: activate_osd: ceph_args+=' --pid-file=td/test-erasure-eio/$name.pid' 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:856: activate_osd: ceph_args+=' --osd-max-object-name-len=460' 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:857: activate_osd: ceph_args+=' --osd-max-object-namespace-len=64' 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:858: activate_osd: ceph_args+=' --enable-experimental-unrecoverable-data-corrupting-features=*' 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:859: activate_osd: ceph_args+=' --osd-mclock-profile=high_recovery_ops' 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:860: activate_osd: ceph_args+=' ' 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:861: activate_osd: ceph_args+= 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:862: activate_osd: mkdir -p td/test-erasure-eio/2 2026-02-19T13:21:48.463 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:864: activate_osd: echo start osd.2 2026-02-19T13:21:48.464 INFO:tasks.workunit.client.0.trial111.stdout:start osd.2 2026-02-19T13:21:48.464 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:865: activate_osd: ceph-osd -i 2 --fsid=477d34f8-4965-4aed-8863-7f0e5a111d9a --auth-supported=none --mon-host=127.0.0.1:7112 --osd_mclock_override_recovery_settings=true --osd-failsafe-full-ratio=.99 --osd-journal-size=100 --osd-scrub-load-threshold=2000 --osd-data=td/test-erasure-eio/2 --osd-journal=td/test-erasure-eio/2/journal --chdir= --run-dir=td/test-erasure-eio '--admin-socket=/tmp/ceph-asok.65583/$cluster-$name.asok' --debug-osd=20 '--log-file=td/test-erasure-eio/$name.log' '--pid-file=td/test-erasure-eio/$name.pid' --osd-max-object-name-len=460 --osd-max-object-namespace-len=64 '--enable-experimental-unrecoverable-data-corrupting-features=*' --osd-mclock-profile=high_recovery_ops 2026-02-19T13:21:48.465 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: cat td/test-erasure-eio/2/whoami 2026-02-19T13:21:48.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:867: activate_osd: '[' 2 = 2 ']' 2026-02-19T13:21:48.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: ceph osd dump --format=json 2026-02-19T13:21:48.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: jq '.flags_set[]' 2026-02-19T13:21:48.465 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:870: activate_osd: grep -q '"noup"' 2026-02-19T13:21:48.475 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:48.475+0000 7f9516c33900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:48.476 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:48.476+0000 7f9516c33900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:48.477 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:48.477+0000 7f9516c33900 -1 WARNING: all dangerous and experimental features are enabled. 2026-02-19T13:21:48.667 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:48.667+0000 7f9516c33900 -1 Falling back to public interface 2026-02-19T13:21:48.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:873: activate_osd: wait_for_osd up 2 2026-02-19T13:21:48.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:969: wait_for_osd: local state=up 2026-02-19T13:21:48.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:970: wait_for_osd: local id=2 2026-02-19T13:21:48.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:972: wait_for_osd: status=1 2026-02-19T13:21:48.687 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i=0 )) 2026-02-19T13:21:48.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:48.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 0 2026-02-19T13:21:48.688 INFO:tasks.workunit.client.0.trial111.stdout:0 2026-02-19T13:21:48.688 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:48.689 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:21:48.905 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:976: wait_for_osd: sleep 1 2026-02-19T13:21:49.078 INFO:tasks.workunit.client.0.trial111.stderr:2026-02-19T13:21:49.078+0000 7f9516c33900 -1 osd.2 34 log_to_monitors true 2026-02-19T13:21:49.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i++ )) 2026-02-19T13:21:49.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:973: wait_for_osd: (( i < 300 )) 2026-02-19T13:21:49.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:974: wait_for_osd: echo 1 2026-02-19T13:21:49.907 INFO:tasks.workunit.client.0.trial111.stdout:1 2026-02-19T13:21:49.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: ceph osd dump 2026-02-19T13:21:49.907 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:975: wait_for_osd: grep 'osd.2 up' 2026-02-19T13:21:50.130 INFO:tasks.workunit.client.0.trial111.stdout:osd.2 up in weight 1 up_from 43 up_thru 19 down_at 35 last_clean_interval [19,34) [v2:127.0.0.1:6818/4090014519,v1:127.0.0.1:6819/4090014519] [v2:127.0.0.1:6820/4090014519,v1:127.0.0.1:6821/4090014519] exists,up 6e401355-bfd0-4449-815d-9cf91494b962 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:978: wait_for_osd: status=0 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:979: wait_for_osd: break 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:982: wait_for_osd: return 0 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:178: rados_put_get_data: wait_for_clean 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1647: wait_for_clean: local cmd= 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1648: wait_for_clean: local num_active_clean=-1 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1649: wait_for_clean: local cur_active_clean 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: get_timeout_delays 90 .1 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: shopt -q -o xtrace 2026-02-19T13:21:50.131 INFO:tasks.workunit.client.0.trial111.stderr:///home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: echo true 2026-02-19T13:21:50.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1593: get_timeout_delays: local trace=true 2026-02-19T13:21:50.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: true 2026-02-19T13:21:50.132 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1594: get_timeout_delays: shopt -u -o xtrace 2026-02-19T13:21:50.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: delays=('0.1' '0.2' '0.4' '0.8' '1.6' '3.2' '6.4' '12.8' '15' '15' '15' '15' '4.5') 2026-02-19T13:21:50.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1650: wait_for_clean: local -a delays 2026-02-19T13:21:50.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1651: wait_for_clean: local -i loop=0 2026-02-19T13:21:50.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1653: wait_for_clean: flush_pg_stats 2026-02-19T13:21:50.211 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2253: flush_pg_stats: local timeout=300 2026-02-19T13:21:50.211 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ceph osd ls 2026-02-19T13:21:50.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2255: flush_pg_stats: ids='0 2026-02-19T13:21:50.430 INFO:tasks.workunit.client.0.trial111.stderr:1 2026-02-19T13:21:50.430 INFO:tasks.workunit.client.0.trial111.stderr:2 2026-02-19T13:21:50.430 INFO:tasks.workunit.client.0.trial111.stderr:3' 2026-02-19T13:21:50.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2256: flush_pg_stats: seqs= 2026-02-19T13:21:50.430 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:50.430 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.0 flush_pg_stats 2026-02-19T13:21:50.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=25769803784 2026-02-19T13:21:50.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 25769803784 2026-02-19T13:21:50.513 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784' 2026-02-19T13:21:50.514 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:50.514 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.1 flush_pg_stats 2026-02-19T13:21:50.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=55834574855 2026-02-19T13:21:50.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 55834574855 2026-02-19T13:21:50.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574855' 2026-02-19T13:21:50.592 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:50.592 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.2 flush_pg_stats 2026-02-19T13:21:50.669 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=184683593731 2026-02-19T13:21:50.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 184683593731 2026-02-19T13:21:50.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-184683593731' 2026-02-19T13:21:50.670 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2257: flush_pg_stats: for osd in $ids 2026-02-19T13:21:50.670 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: ceph tell osd.3 flush_pg_stats 2026-02-19T13:21:50.746 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2258: flush_pg_stats: seq=111669149702 2026-02-19T13:21:50.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2259: flush_pg_stats: test -z 111669149702 2026-02-19T13:21:50.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2263: flush_pg_stats: seqs=' 0-25769803784 1-55834574855 2-184683593731 3-111669149702' 2026-02-19T13:21:50.747 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:50.747 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 0-25769803784 2026-02-19T13:21:50.747 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:50.748 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=0 2026-02-19T13:21:50.749 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 0-25769803784 2026-02-19T13:21:50.749 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:50.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=25769803784 2026-02-19T13:21:50.750 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.0 seq 25769803784' 2026-02-19T13:21:50.750 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.0 seq 25769803784 2026-02-19T13:21:50.750 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:50.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803782 -lt 25769803784 2026-02-19T13:21:50.971 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2271: flush_pg_stats: sleep 1 2026-02-19T13:21:51.972 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2272: flush_pg_stats: '[' 300 -eq 0 ']' 2026-02-19T13:21:51.972 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 0 2026-02-19T13:21:52.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 25769803784 -lt 25769803784 2026-02-19T13:21:52.195 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:52.196 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 1-55834574855 2026-02-19T13:21:52.196 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:52.197 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=1 2026-02-19T13:21:52.198 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 1-55834574855 2026-02-19T13:21:52.198 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:52.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=55834574855 2026-02-19T13:21:52.199 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.1 seq 55834574855 2026-02-19T13:21:52.199 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.1 seq 55834574855' 2026-02-19T13:21:52.200 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 1 2026-02-19T13:21:52.418 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 55834574856 -lt 55834574855 2026-02-19T13:21:52.418 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:52.419 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 2-184683593731 2026-02-19T13:21:52.419 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:52.419 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=2 2026-02-19T13:21:52.420 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 2-184683593731 2026-02-19T13:21:52.420 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:52.421 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=184683593731 2026-02-19T13:21:52.421 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.2 seq 184683593731 2026-02-19T13:21:52.422 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.2 seq 184683593731' 2026-02-19T13:21:52.422 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 2 2026-02-19T13:21:52.643 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 184683593731 -lt 184683593731 2026-02-19T13:21:52.644 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2266: flush_pg_stats: for s in $seqs 2026-02-19T13:21:52.644 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: echo 3-111669149702 2026-02-19T13:21:52.644 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: cut -d - -f 1 2026-02-19T13:21:52.645 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2267: flush_pg_stats: osd=3 2026-02-19T13:21:52.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: echo 3-111669149702 2026-02-19T13:21:52.646 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: cut -d - -f 2 2026-02-19T13:21:52.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2268: flush_pg_stats: seq=111669149702 2026-02-19T13:21:52.647 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2269: flush_pg_stats: echo 'waiting osd.3 seq 111669149702' 2026-02-19T13:21:52.647 INFO:tasks.workunit.client.0.trial111.stdout:waiting osd.3 seq 111669149702 2026-02-19T13:21:52.647 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: ceph osd last-stat-seq 3 2026-02-19T13:21:52.869 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2270: flush_pg_stats: test 111669149703 -lt 111669149702 2026-02-19T13:21:52.869 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: get_num_pgs 2026-02-19T13:21:52.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:52.870 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:53.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1654: wait_for_clean: test 5 == 0 2026-02-19T13:21:53.170 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1658: wait_for_clean: true 2026-02-19T13:21:53.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: get_num_active_clean 2026-02-19T13:21:53.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1355: get_num_active_clean: local expression 2026-02-19T13:21:53.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1356: get_num_active_clean: expression+='select(contains("active") and contains("clean")) | ' 2026-02-19T13:21:53.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1357: get_num_active_clean: expression+='select(contains("stale") | not)' 2026-02-19T13:21:53.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1358: get_num_active_clean: ceph --format json pg dump pgs 2026-02-19T13:21:53.170 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1359: get_num_active_clean: jq '.pg_stats | [.[] | .state | select(contains("active") and contains("clean")) | select(contains("stale") | not)] | length' 2026-02-19T13:21:53.388 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1662: wait_for_clean: cur_active_clean=5 2026-02-19T13:21:53.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: get_num_pgs 2026-02-19T13:21:53.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: ceph --format json status 2026-02-19T13:21:53.389 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1416: get_num_pgs: jq .pgmap.num_pgs 2026-02-19T13:21:53.683 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: test 5 = 5 2026-02-19T13:21:53.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1663: wait_for_clean: break 2026-02-19T13:21:53.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:1678: wait_for_clean: return 0 2026-02-19T13:21:53.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:374: TEST_rados_get_with_subreadall_eio_shard_1: delete_erasure_coded_pool pool-jerasure 2026-02-19T13:21:53.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:84: delete_erasure_coded_pool: local poolname=pool-jerasure 2026-02-19T13:21:53.684 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:85: delete_erasure_coded_pool: ceph osd pool delete pool-jerasure pool-jerasure --yes-i-really-really-mean-it 2026-02-19T13:21:53.949 INFO:tasks.workunit.client.0.trial111.stderr:pool 'pool-jerasure' does not exist 2026-02-19T13:21:53.961 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:86: delete_erasure_coded_pool: ceph osd erasure-code-profile rm myprofile 2026-02-19T13:21:54.255 INFO:tasks.workunit.client.0.trial111.stderr:erasure-code-profile myprofile does not exist 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/erasure-code/test-erasure-eio.sh:42: run: teardown td/test-erasure-eio 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs= 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:21:54.267 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:21:54.374 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:21:54.375 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:21:54.376 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:21:54.376 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:21:54.377 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:21:54.378 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:21:54.378 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:21:54.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:54.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:21:54.379 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:21:54.380 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:54.380 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:21:54.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o '' = 1 ']' 2026-02-19T13:21:54.381 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:21:54.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:21:54.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:54.391 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:54.391 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:21:54.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:21:54.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:21:54.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2401: main: code=0 2026-02-19T13:21:54.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2405: main: teardown td/test-erasure-eio 0 2026-02-19T13:21:54.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:155: teardown: local dir=td/test-erasure-eio 2026-02-19T13:21:54.393 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:156: teardown: local dumplogs=0 2026-02-19T13:21:54.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:157: teardown: kill_daemons td/test-erasure-eio KILL 2026-02-19T13:21:54.394 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: shopt -q -o xtrace 2026-02-19T13:21:54.394 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: echo true 2026-02-19T13:21:54.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:336: kill_daemons: local trace=true 2026-02-19T13:21:54.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: true 2026-02-19T13:21:54.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:337: kill_daemons: shopt -u -o xtrace 2026-02-19T13:21:54.394 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:353: kill_daemons: return 0 2026-02-19T13:21:54.395 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: uname 2026-02-19T13:21:54.395 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:158: teardown: '[' Linux '!=' FreeBSD ']' 2026-02-19T13:21:54.396 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: stat -f -c %T . 2026-02-19T13:21:54.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:159: teardown: '[' ext2/ext3 == btrfs ']' 2026-02-19T13:21:54.397 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:162: teardown: local cores=no 2026-02-19T13:21:54.398 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: sysctl -n kernel.core_pattern 2026-02-19T13:21:54.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:163: teardown: local pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:54.398 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:165: teardown: '[' / = '|' ']' 2026-02-19T13:21:54.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: grep -q '^core\|core$' 2026-02-19T13:21:54.399 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: dirname /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-19T13:21:54.399 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:171: teardown: ls /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:21:54.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:180: teardown: '[' no = yes -o 0 = 1 ']' 2026-02-19T13:21:54.400 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:189: teardown: rm -fr td/test-erasure-eio 2026-02-19T13:21:54.402 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: get_asok_dir 2026-02-19T13:21:54.402 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:99: get_asok_dir: '[' -n '' ']' 2026-02-19T13:21:54.402 INFO:tasks.workunit.client.0.trial111.stderr://home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:102: get_asok_dir: echo /tmp/ceph-asok.65583 2026-02-19T13:21:54.402 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:190: teardown: rm -rf /tmp/ceph-asok.65583 2026-02-19T13:21:54.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:191: teardown: '[' no = yes ']' 2026-02-19T13:21:54.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:198: teardown: return 0 2026-02-19T13:21:54.403 INFO:tasks.workunit.client.0.trial111.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/ceph-helpers.sh:2406: main: return 0 2026-02-19T13:21:54.404 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-19T13:21:54.404 DEBUG:teuthology.orchestra.run.trial111:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-19T13:21:54.473 INFO:tasks.workunit:Stopping ['erasure-code'] on client.0... 2026-02-19T13:21:54.473 DEBUG:teuthology.orchestra.run.trial111:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-19T13:21:54.854 DEBUG:teuthology.parallel:result is None 2026-02-19T13:21:54.855 DEBUG:teuthology.orchestra.run.trial111:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T13:21:54.881 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T13:21:54.881 DEBUG:teuthology.orchestra.run.trial111:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-19T13:21:54.939 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-19T13:21:54.939 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-19T13:21:54.951 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-19T13:21:54.952 DEBUG:teuthology.orchestra.run.trial111:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-19T13:21:55.107 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-19T13:21:55.119 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-19T13:21:55.119 DEBUG:teuthology.orchestra.run.trial111:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-19T13:21:55.137 INFO:teuthology.orchestra.run.trial111.stderr:bash: line 1: ntpq: command not found 2026-02-19T13:21:55.161 INFO:teuthology.orchestra.run.trial111.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-19T13:21:55.161 INFO:teuthology.orchestra.run.trial111.stdout:=============================================================================== 2026-02-19T13:21:55.161 INFO:teuthology.orchestra.run.trial111.stdout:^+ 172-235-154-118.ip.linod> 4 6 377 42 +211us[ +164us] +/- 53ms 2026-02-19T13:21:55.161 INFO:teuthology.orchestra.run.trial111.stdout:^* daisy.zia.io 2 6 377 40 +1032us[ +985us] +/- 44ms 2026-02-19T13:21:55.161 INFO:teuthology.orchestra.run.trial111.stdout:^+ us-east-2.clearnet.pw 2 6 377 107 -832us[ -926us] +/- 67ms 2026-02-19T13:21:55.161 INFO:teuthology.orchestra.run.trial111.stdout:^+ 172-233-177-198.ip.linod> 2 6 377 41 -1169us[-1216us] +/- 84ms 2026-02-19T13:21:55.162 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-19T13:21:55.172 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-19T13:21:55.172 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-19T13:21:55.190 DEBUG:teuthology.orchestra.run.trial111:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-19T13:21:55.245 DEBUG:teuthology.orchestra.run.trial111:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e 'comm="systemd".*denied.*\{ prog_run \}.*tclass=bpf.*permissive=1' 2026-02-19T13:21:55.303 INFO:teuthology.orchestra.run.trial111.stderr:grep: Invalid content of \{\} 2026-02-19T13:21:55.311 DEBUG:teuthology.orchestra.run:got remote process result: 2 2026-02-19T13:21:55.311 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-19T13:21:55.323 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-19T13:21:55.339 INFO:teuthology.task.internal:Duration was 883.893569 seconds 2026-02-19T13:21:55.340 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-19T13:21:55.356 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-19T13:21:55.356 DEBUG:teuthology.orchestra.run.trial111:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-19T13:21:55.399 INFO:teuthology.orchestra.run.trial111.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-19T13:21:55.661 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-19T13:21:55.661 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial111.front.sepia.ceph.com 2026-02-19T13:21:55.662 DEBUG:teuthology.orchestra.run.trial111:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-19T13:21:55.722 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-19T13:21:55.722 DEBUG:teuthology.orchestra.run.trial111:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T13:21:55.803 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-19T13:21:55.804 DEBUG:teuthology.orchestra.run.trial111:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-19T13:21:55.870 INFO:teuthology.orchestra.run.trial111.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-19T13:21:55.871 INFO:teuthology.orchestra.run.trial111.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-19T13:21:55.872 INFO:teuthology.orchestra.run.trial111.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-19T13:21:55.872 INFO:teuthology.orchestra.run.trial111.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-19T13:21:55.872 INFO:teuthology.orchestra.run.trial111.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-19T13:21:55.874 INFO:teuthology.orchestra.run.trial111.stderr: 87.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-19T13:21:55.876 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-19T13:21:55.892 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-19T13:21:55.892 DEBUG:teuthology.orchestra.run.trial111:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-19T13:21:55.943 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-19T13:21:55.959 DEBUG:teuthology.orchestra.run.trial111:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:21:56.027 INFO:teuthology.orchestra.run.trial111.stdout:kernel.core_pattern = core 2026-02-19T13:21:56.040 DEBUG:teuthology.orchestra.run.trial111:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-19T13:21:56.057 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-19T13:21:56.058 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-19T13:21:56.074 INFO:teuthology.task.internal:Transferring archived files... 2026-02-19T13:21:56.076 DEBUG:teuthology.misc:Transferring archived files from trial111:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/nmordech-2026-02-19_12:10:46-rados-wip-rocky10-branch-of-the-day-2026-02-18-1771415652-tentacle-distro-default-trial/58710/remote/trial111 2026-02-19T13:21:56.078 DEBUG:teuthology.orchestra.run.trial111:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-19T13:21:56.137 INFO:teuthology.task.internal:Removing archive directory... 2026-02-19T13:21:56.137 DEBUG:teuthology.orchestra.run.trial111:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-19T13:21:56.186 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-19T13:21:56.204 INFO:teuthology.task.internal:Not uploading archives. 2026-02-19T13:21:56.204 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-19T13:21:56.221 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-19T13:21:56.221 DEBUG:teuthology.orchestra.run.trial111:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-19T13:21:56.244 INFO:teuthology.orchestra.run.trial111.stdout: 786901 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 19 13:21 /home/ubuntu/cephtest 2026-02-19T13:21:56.245 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-19T13:21:56.256 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-19T13:21:56.293 INFO:teuthology.run:Summary data: description: rados/standalone/{supported-random-distro$/{rpm_latest} workloads/erasure-code} duration: 883.8935685157776 flavor: default owner: scheduled_nmordech@soko04.front.sepia.ceph.com success: true 2026-02-19T13:21:56.294 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-19T13:21:56.428 INFO:teuthology.run:pass